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ABSTRACT 



The occurrence ana detection of software errors was stu- 
died in four software orojects. Errors were analyzed with 
respect to complexity measures of individual subroutines. 
This paper also provices definitions of error categories and 
types. The detailed documentation of the software produc- 
tion effort allowed both a qualitative and Quantitative 
analysis of software errors with respect to error types. 
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INTRODUCTION 



In a recent SHARE study the annual growth rate of 
software demands over the years 1957 - 1985 was estimated to 
be as high as 21-23% (15]. The same study indicates that 
software production may develop at a growth rate of only 
11.5-17%. 

Besides this quantitative a a o / Boehm [7] ooints out the 
following Qualitative deficiencies: Since software is in- 
creasingly used for important functions such as defense sys- 
tems/ traffic control systems/ and medical ourooses/ 
software is needed which can be trusted. Although 80% of 
the money used for the production of the Apollo Manned 
Spaceflight Program was devoted to testing [121/ software 
errors occurred in Aoollo 8 , 11/ and 14. One critical error 
even occurred during the lunar landing onase. 

Software engineering (for definition see Chapter II) may 
be accepted as the method which yields the answer to tne 
problem of producing reliable software at reasonable cost 
[71. Little is known about software errors besides their 
existence and their negative influence on cost and perfor- 
mance. Increasing difficulties to produce sufficient 
amounts of high duality software make it mandatory to invent 
software development methods which are designed to avoid the 
most common software errors/ achieve easier testing and 
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debugging and improve methods of error detection. 

This cannot be aone without detailed studies about the 
occurrence of software errors. Only this knowledge will 
helo to identify the nature of errors ana finally lead to 
improvements of software methodoloay. Other motivating fac- 
tors for this study were: 

1. Lack of currently available data to supoort analysis 

and research in the area of error ocurrence and 
detection. 

2. The need to examine existing software error models. 

The objective of the experiment was to gather accurate 
and complete information about errors, their occurrence and 
detection throughout the entire process of software develop- 
ment. Limitations of the experiment were the relatively 
small number of projects and tne lack of exoer i enced pro- 
grammers. The use of the experimenter as the subject pro- 
grammer ana evaluator introduced Diases also. The scope of 
the experiment did not include the Program maintenance 
phase. It was felt to be of great importance to record all 
errors, even trivial errors which could hurt the image of 
the programmer. A detailed description, of the experiment 
and the error data is given in chapters 4 thru 5 and in the 
append ices. 
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II 



CONCEPTS AND KEYWORDS 



Communication among computer scientists is hindered by 
the lack of common terminolooy. In order to avoid misunder- 
standing, the most important keywords will be defined in 
this chaoter. Wherever possible definitions are adooted 
from other publications. 



A. SOFTWARE ENGINEERING TERMS 



1. Software Engineering 

"The oractical application of scientific knowledge 
in the design and construction of computer programs and 
the associated doc umen t a t i on reauired to develop, 
operate and maintain them", T7T 



2 , Top-Down Design 

A program design method which starts at a very gen- 
eral level Dy identifying major functions and Proceeds 
stepwise to lower levels to the identification of lesser 
functions that derive from the major ones, "Top-down 
design” and "hierarchical program design" are equivalent 



terms 



[121 



3 . Structured Programming 

Although this term is used in numerous Publications 
its understanding varies because of its conceptual na- 
ture. As proposed by Dijkstra as early as 1965 [30], it 
first was a conceot to eliminate GO TO statements. 
Later on it was combined with the idea of top-down 
design, [31] 

Rather than providinq a formal definition, Yourdan 
[12] and Auerbach [27] describe structured programming 
Dy its objectives: " increase of readability" and "de- 
crease of testing problems." The following definition 
of structured programming as it applies to this paper is 
derived from Kirchgaessner’s description. [28] 



Structured programming is a method of developing 
programs for which the underlying concept is a top-down 
design which eventually leads to a modular structure of 
the program. All algorithms are designed using the five 
major constructs allowed in structured programming: 

- Sequential statement 

- IF THEN ELSE statement 

- M H I L E loop (conditional 1 o o d ) 

- FOR loop (iterative loop) 

- CASE statement 



i t 



be 



In addition to this definition 



should 



emphasized that structured programming is a design dis- 
cipline rather than a way of writing code. Therefore 
it should not be defined as " coding without GO TO 
statements." Methods have been developed to implement 
structured designs in FORTRAN. (12) However/ it is 
easier/ and more readable to use a block structured 
language/ such as ALGOL/ to imolement a structured 
design. 

4 . Directed Graph 

"A directed graph is a geometric graph/ consisting 
of nodes and arcs with a direction of traversal associ- 
ated with each arc." [16] 

5 . Module 

"A module is a Physical combination of program 
instructions that is independent of others with respect 
to compiling/ assembling and loading and which performs 
a specific function." [16] "Each module has a small 
number of interactions with other modules." [4] 

b . R roar am 

A proaram is a set of integrated modules. (16] 

7. Software Development Process 

The software development process includes the 
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following st ages 



- Problem Analysis 

- Design and Design Review 

- Coding 

- Debugging 

- Testing 

- Inteo ration 

• Imol enfientat ion 

- Maintenance 

Despite minor differences with other publications 
such as Auerbach (21 the above view of the software 
development process will be adopted as the definition of 
the software development process for this thesis. 
Althouah the development staaes are listed sequentially 
it should be noted that software production in general 
is viewed as a dynamic process including feedbacks to 
earlier stages of a project. Thus none of these phases 
may be considered to be completed when the subsequent 
phase has started. In some projects/ testing itself is 
preceded by activities such as the writing of test 
specifications ana test procedures. Figure 1 indicates 
oossiole feedback situations within a complex program 



development Process 
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IMPLEMENTATION 



FIGURE 1 

ThE FEEDBACK CYCLES DURING A SOFTWARE DEVELOPMENT PROJECT 
8. Common Data 

Common Data is data which is used by several 



modu 1 e s 



B 



TESTING, DEBUGGING AND ERRORS 



1 . Debugging 

(a) Debugging is the action to check subroutines and 
modules as to whether or not they Derform according 
to orogrammer exoect at i ons , It is also the action 
which one takes to locate and correct known errors. 

(b) rest i na is the action taken after modules have 
been integrated to check whether or not the program 
meets specifications. As statea by Dijkstra [29], 
testing can only establish the presence of errors, 
not their absence. 

2. Software Error 

A software error is a mistake made during software 
development, which leads to an incorrect action or 
result with resoect to the proaram specifications within 
the program. [ 1 o ] Errors made in the specification 
phase of the program, hardware errors and compiler er- 
rors Ci.e. errors within the compiler program) were not 
studied durina this experiment. Categorization and 
definitions of software errors with respect to their 
occurrence are given in chaoter 3. 
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c 



software reliability and correctness 



1. Software Reliability 

" Software reliability is the probability that a corn- 
outer orogram will oerform its (user) intended function 
for a specified time interval under stated operating 
conditions." [17] 



2. Software Correctness 

"The feature of software that renders it operation- 
ally useful for its intended functions is its correct- 
Correct software does the thinas intended by 



ness . 
desi an • 



[R] 



- 1 6 



Ill, DEFINITION OF ERROR CATEGORIES AND TYPES 



When analyzing the causes of errors in software produc- 
tion there is an onvious need to precisely describe an error 
in accordance with its category and tyoe. One of the objec- 
tives of this study was to identify errors with respect to 
their occurrences ana causes. 

It was was found that many errors in software oroduction 
belong to one of the following major categories: 

1. Problem Specification Errors 

2. System Design Errors 

3. Proqram Desian Errors 
4 • Codinq Errors 

5. Clerical Errors 
o. Debugging Errors 
7. Testing Errors 

Implementation Errors 



A. DEFINITONS OF ERROR CATEGORIES 



l. Proolem Specification Error 

Any mistake or deficiency w 
analysis of the proaram apoli cation 
software requirements with resoect 



h i c h 


occurs i n 


the 


or in 


spec i tying 


tne 


to the 


i ntended 


pro- 



o r 



gram 



application such as incomplete 



e r roneous 



ambiguous statements 



2m System Design Error 

An error mace in the t ransforrnat i on of the user- 
originated program snec i f i cat i ons into systems design 
spec i f i cat i ons . 

3* Program Design Error 

A orogram design error is one made in the transfor- 
mation of the system design into specific algorithms ana 
da t a s t rue t u res . 

4 . Cod i ng Error 

A coding error is one which is maae during the 
transformation of a program design into source language* 

5. Clerical Error 

A clerical error is human failure which is made dur- 
ing the transformation of any ohysical representation 
(such as a coding sheet) of tne orogram or parts of the 
program to another physical represen t at i on (such as 
ounch cards). 

6. Debuaging Error 

InaooroDriate use of deougginq tools, insufficient 
or i naooropri ate selection of test cases, test data or 
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misinterpretation of deougaing results. 

7 . Testinq Error 

Inappropriate or insufficient test cases or test 
data# misunderstanding of functional requirements of the 
program# deviation from test olan or misinterpretation 
of test results. 

8 . Implementation Error 

Unexpected environmental oroclem which occurs during 
the implementation of a program which could not have 
been anticipated in a orevious staqe of the software 
development. This includes cases such as changes of 
haraware by the manufacturer and performance problems 
with operat inq systems. 

Accorai no to the cef ini t ion of software errors# an error 
which aoes not lead to an incorrect action or result within 
the program cannot be identified as a software error. For 
example insufficient test cases during debugging of a 
subroutine may not lead to a software error. however# the 
potential exists for undetected software errors when test 
cases are insufficient. According to the definition of 
debuogi na errors# errors of this category do not necessarily 
affect the program. Therefore debugging errors will not be 



detected very often; 



however# because of being a potential 



error source* errors of this category were considered to be 
important* During the experiment described in chapter 4 all 
debugging errors were recorded* Similarly design errors, 
which were detected during a design review/or faulty com- 
ments were recorded. This was of great importance* since it 
yields significant information about error sources* 

B* definition OF ERROR TYPES 

The list of error type definitions given below identi- 
fies the most common types of errors within the major ca- 
tegories of software errors. If approcri ate, examples are 
given for clarity. 

These definitions are important because it has been 
discovered that without a definition of error types* errors 
of the same tyoe could be recorded in different ways* not 
only by different programmers* but also by the same person 
at different times* 

1 • Design Errors 

The following types of errors apply to both ca- 
teoories "System Design Errors" ana "Program Design 
Errors" : 

Dl : Communication Error 

An error occurs due to improper communication 
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between members of a programming team or due to 
inaDDropriate systems design specifications. Typi- 
cal cases are conflicting interpretation of common 
data and misunderstandina of module functions. 

D 2 : Design Negligence 

p arts of program or system design spec i feat ions are 
neglected or necessary cooperation with the 
originator(s) of the specification have been omit- 
ted# such as neglecting desired output formats or 
lack of documentation. 

Di : Forgotten Cases or Steos 

while solving the problem one or more cases have 
not been considered, such as neglecting leap years 
for calculation of dates or necessary step(s) to 
solve the problem have been omitted, such as deter- 
mining the status of peripheral devices before open- 
ing I/O, or forgotten conversion from decimal to 
binary or vice versa. 

D4 : Timing Problems 

Misconception in scheduling program events, such as 
assianinq inappropriate priorities or changing data 
before completion of I/O. 

Db : Errors in I/O Concepts 

Errors made in the design of input or output, such 
as mi sunderstandi ng of hardware requirements of I/O 
devices or channels. Typical cases are 
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misunderstanding of the capabilities of terminals or 
graphics devices# neglecting the size of lines on a 
1 i nepri nter or the interrupt capabilities of peri- 
pheral devices. 

Ob : Data Design Error 

The data design does not fit tne needs of the pro- 
gram. Typical examples are wrona sizes of arrays/ 
buffers or freelists. Also cases of inappropriate 
data definitions/ such that the chosen data types do 
not accomplish the desired precision. This error 
type also includes design of i naopropr i ate I/O for- 
mats and improper design of common data. 

0/ : Initialzation Error 

/Jrong or incomplete initialization such as forgotten 
initialization of global pointers/ arrays or flag 
variables or forgotten initialization of interrupt 
h and 1 e r s . 

D b : Inadequate Checkina 

Checking of variables or input data is incomplete or 
wrong according to program specifications. 

DV : Extreme Conditions Neglected 

Data or macnine dependent extreme conditions are 
neglected/ such as numeric values which cause un- 
derflow/ overflow/ excess of array limits or maximum 
values of real numbers or integers# ana occurrence 
of zero or negative values due to round off. 
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010: Seauencing Error 

Program events or decisions scheduled in wrong se- 
quence such as transformation from problem statement 
into a sequence of operations. 

Dll: Indexing Error 

Faulty index calculation such as designing an er- 
roneous algorithm for accessing array elements or 
items of a table. 

DIP: Looo Control Errors 

Error in controlling either an iterative or condi- 
tional repetitive algorithm/ such as basing exit or 
break decisions on conditions which can never occur. 
013: Misuse of Boolean Expression 

An error is made in usina Boolean operators/ such as 
constructing a false inverse of an expression. A 
trivial examole of this error tyoe is solving 
( x + y ) ' * z = ( x ' + y 1 ) * z 

instead of: 

( x + y ) * * z = ( x • * y ' ) * z 

where +/ */ * are the Boolean operators " OR" / "AND" / 

and "NOT" respectively. 

01 '4 : Mathematical Error 

The mathematical solution of a problem within the 
programming project does not or not always represent 
the correct result according to the proqram specifi- 
cations. This includes cases in which a 
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mathematical equation is simply wrong, like a wrong 
formula for calculating a mean or variance of given 
set of numbers and cases in which the mathematical 
solution does not aool y for all possible incuts, 
such as solving a square root for negative expres- 
sions. 

D15: Representation Error 

An error is made in the process of the physical 
reoresen t at i on of thoughts/ such as writing desiqn 
documentation different from what the designer had 
i n t ended . 

D16: M i sunde r s t and i ng of Proolem Specifications 

Error resulting from 'misunderstanding the problem 
specifications. For example a decision table/ 
developed during design/ includes some undesired 
combinations of actions. 

017: Other Design Errors 

Any other design error which is not one of the types 
1 i stea above . 



Coding Errors 

Cl : M i sunde r s t ana i ng of Design 

An error made in interpreting the desiqn specifica- 



tions/ 



such as m i sunde rs t ana i ng an algorithm/ data 



descriptions or critical terms* such as mathematical 
expressions which leads to an error in implementa- 
tion of the underlying a e s i g n . 

C2 : Negligence 

Parts of the design are neglected or necessary 



cooperat i on 


with 


the 


designer(s) i 


s omitted. 


This 


will 


include 


cases 


w h e 


re proa ramme r s 


are uncertain 


about 


the 


design 


o r 


parts of it 


and fail to 


take 



apDroor i ate action* such as requesting an explana- 
tion from the designer (s) or obtaining the missing 
information from other sources such as project 
specifications* hardware or software manuals. 

C3 : I/O Format Error 



Use of improper I/O formats in coding* such as using 
formatted I/O when nonformatted I/O is specified or 
using inappropriate record lengths. 



C4 : 


M i s d 1 a c e d 


Data Declaration 






E r r o r 


made 


i n 


declaring data 


either i 


n the wrong 


b 1 oc k 


level 


( w h i 


1 e coding in 


a block 


s t rue t u red 


1 anguaae ) or 


i n 


an i napo r oo r i ate 


Dosi t ion 


within the 



source coae. 

C5 : Multiple Data Declaration 

'Multiple declarations of same data. 
Cg : Missing Data Declaration 

Necessary data declaration left out. 
C7 : Inadequate Data 
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Error made in choosing size or tyoe of data, such as 
decl arinq single precision instead of double prec i - 
sion, 

C 3 : Initialization Error 

Wrong or forgotten initialization. 

C9 : Error in Parameter Passing 

Passing of parameters is incomplete, wrong or types 
do not match. 

CIO; Inadeauate or Forgotten Checking 

Checking of inout is either incomplete or wrong 
according to program design specifications, such as 
not checkina for invalid input within a subroutine 
which receives characters from a user terminal. A 
typical case of this error tyoe occurs when numeri- 
cal inout is expected and is converted to binary 
without checkinq the validity of the input charac- 
ter. Another case is an inadequate check of the 
size of the input number. 

C 1 1 : Level Problems 

While coding in a block structured lanquage an error 
results from confusinq block levels, such as 
misunderstanding the scoce of variables, e.g. an 
ALGOL programmer tries to use variables or data of 
a lower level within the main level (level 1) of a 
D r oq ram. 

C 1 2 : M i s s i n g Declarations of Block Limits 
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d h i 1 e coding in a bloc* structured language mandato- 
ry declarations of bloc* level limits such as BEGIN r 
END are left cut. 

C 1 3 : Case selection error 

An error is made in connection with a CASE statement 
or comouted GO TO statement or an equivalent con- 
struct. For examole in an imolementation of a GO TO 
SWITCH in C V S - 2 an error occurs due to wrong se- 
quence of labels. C M S - 2 is a programming language 
used for tactical programming within military appli- 
cations [361 . 

C 1 4 : GO TO Problems 

An error is made in connection with the use of a GO 
TO statement. 

CIS: Comment Error 

Stating a faulty or misleading comment. 

C16: Forgotten Delimiter 

Delimiter left out. 

C17: Inconsistency in Naming 

An error is made when naming an identifier in dif- 
ferent ways. For example a subroutine declared as 
GETCHAR is called by using GETCH. 

C18: Wrona Use of Nested IF Statements 

An error is made during the construction of a nested 
IF statement. Errors of this type occur in 
transforming a flowchart or decision table logic 
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into nested IF statements or using an incorrect for- 
mat which will cause a compile error or associating 
any of the ELSE cases with a wrong IF clause. 

C 1 9 : Indexing Error 

Indexing is not aoorooriate or left out. 

C 2 0 : Inconsistent Use of Variables or Data 

An error occurs because variables or data are used 
for conflicting purposes. For example usage of a 
temporary storage location in different parts of the 
program coulo oe implemented in such a manner that 
information, is overwritten by some subroutine but 
still be needed by another subroutine. Another 
example is the usage of common variables for dif- 
ferent purposes like pointer and index. 

C21: Sequencing Error 

An improper sequence of statements which leads to an 
error. For example an exchange of array elements is 
p r og rammed 

TEMP A ( i ) 

A ( j ) 4- TEMP 
A ( i ) <- A ( j ) 

which does not accomplish the desired exchange of 
the two el emen t s . 

C22: Flag Usage Problems 

An error is made in using flags in order to control 
program logic. 
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C 2 3 : Syntax Error 



Violation 


o f 


syntax 


rules of 


the programming 


language. 


This 


includes 


e r r o r s in 


connection with 



transformation of boolean exoressions and arithmetic 
expressions. 

C24 : Loop Control Error 

Error in implementing the control structures of an 
iterative or conditional repetitive alaorithm. 

C 2 5 : Incorrect Exit from Subroutines 

An error occurs in the implementation of an exit 
f rom a subrout i ne . 

C26: Language Usace Problems 

An error is made due to idiosyncrasies of the pro- 
gramming 1 ancuaae which are not thoroughly under- 
stood by the programmer. 

C27: Forgotten Statements 

An error is made by leavinq out a necessary state- 
ment which the programmer knew was necessary. 

C28: Rep resen t a t i on Error 

A codino error which is introduced during the pro- 
cess of the physical representation of thoughts/ 
such as writing a statement different from what one 
i n t ended . 

C29 : Control Seauence Error 

An error is made which causes the program to follow 
a wrono branch after a decision. 
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C30: Incorrect Subroutine Usage 

Using the wrong subroutine or using a subroutine 
i naporopri atel y / e.g. a programmer’s expectation of 
what a subroutine will accomplish is different from 
what the subroutine actually does* 

C 3 1 : Other Cooing Errors 

Any other coding error which is is not one of tne 
types listed above. 



3. Clerical Errors 

A 1 : Manual Error 

An error resulting from lack of motoric skill or 
temporary manual disfunction^ such as 

- Errors of Commission (e.g. writing or typing 
”busu" instead of "busy 11 ) 

- Errors of Omission (e.g. writing or typing 
"BOhN" instead of "BROWN") 

Errors of Transposition (e.g. writing or typing 
" h t e M instead of "the") 

A<? : Men t a 1 E r ro r 

An error resulting from a mental misfunction, such 
as 

- Perceptual Errors (e.g. reading " 0 M instead of 

”0”, 11 instead of 11 or "DOG" instead of "DOD") 
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- Expectation Errors ( e . q . in a given context a 
certain word or exoression is exoected and the 
actual word or expression is overlooked, such as 
assuming "FOR I:=N STEP 1 UNTIL M DO" where "FOR 
N : = 1 STEP -1 UNTIL M DO" is the the actual im- 
plementation of an iterative looo in ALGOL W • ) 

A3 : Procedural Errors 

Given a proper assiqnment for a clerical task which 
implies some well defined sequence of actions which 
could be expected from the assignment, some of the 
steps are either forqot ten or not carried out prop- 
erly, such as some lines of code on a coding sheet 
are omitted by the keypunch operator or card correc- 
tion procedures are not followed appropriately or 
inserted cards are misplaced. 

A4 : Other Clerical Errors 

Any other clerical error which is not one of the 
types listed above. 



4. Debuaginq Errors 

81 : Inaoprocri ate Use of Deouqaing Tools 

r o n q selection of debugqina tools, such as traces, 
snapshots or aumos or using them inappropriately. 
For example using a snapshot before instead of after 
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a certain instruction or taking a dump of wrong 
memory locations. 

02 : Insufficient or Inaporooriate Selection of 

Test Cases or Test Data 

The test cases or test data provide insufficient 
test coverage of the proaram or fail to provide 
test i na of critical marts of the oroqram. 

03 : Misinterpretation of Debugging Results 

An inaporooriate action is taken due to misin- 
terpretation of debugging results/ such as a wrong 
result is overlooked or misunderstanding of a 
correct result leads to additional errors. 

B 4 : Misinterpretation of the Error Source 

An inaporooriate action in correcting a discovered 
software error that results from wrong assumotions 
about the error source/ such as correcting the symp- 
toms of an error rather than its cause or the pro- 
grammer fails to foresee the i^oact of a change on 
other parts of the program and inserts additional 
errors into the Program. 

05 : Negligence 

Errors resultina from negligence in debugging. For 
example the implementation of an error correction is 
assumed to be error free without further checking. 

Bo : Other Debuaginq Errors 

Any other aebuggina error which is not one of the 
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types listed above 



5 . Testina Errors 



Tl : Inadeauate Test Case(s) or Test Data 

Selected test caseCs) or test data do not orooerly 
test the orogram . 

Id : Misinterpretation of Test Results 

An inappropriate action is taken due to misunder- 
standing of test results/ such as a wrong result is 
overlooked or misunderstanding of a correct result 
leads to an implementation of additional errors. 

T 3 : Misinterpretation of Problem Specifications 

Error(s) resulting from misunderstanding the program 
soecifi cations/ such as incorrect interpretation of 
user defined functions which leads to an insertion 
of an error or to over looking an existing error. 

T4 : Negligence 

Errors result ina from negligence in testing. For 
examole a tactical real time system is merely tested 
within specified limits and tests fail to examine 
the behavior of the system for cases where limits 
are exceded. In this oarticular example negligence 
can cause serious results in critical situations. 
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T5 : Other Testing Errors 

Any other testing error which is not one of the 
types listed above. 



In order to get meaningful results/ the above defini- 
tions were used throughout the project. This list is not 
considered complete. However/ it permitted identification 
of software error types for the purpose of this experiment. 
Some of the listed error tvoes could be further subdivided 
into subsets of eacn type if this were desired. 

To contribute tc analysis of error sources/ it was 
decided to list the same error type which occurs in dif- 
ferent software development phases under different ca- 
tegories. For example one type of error which has the same 
name (Icon control error) may occur in both design and cod- 
ing phases. however/ the cause of the error could be d i f - 
ferent . 

Error severity is related to the effect of the error on 
system operation. There is no direct relationship between 
error tyoe and severity. For example mistyping of a single 
character of a variable name in FORTRAN can cause strange 
results/ whereas the same error made in a comment mav not 
affect the program at all. The consideration of error 
severity was oeyond the scope of this project. 
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IV 



EXPERIMENT DESCRIPTION 



A. GENERAL REMARKS 

Error data were gathered in four programming projects 
ranging from small to large. During 280 man hours of total 
project time more than 2000 source statements were produced. 
A total of 173 errors were recorded. 

The scooe of the programming projects was limited to the 
following software development phases: 

- Design and Design Review 

- Coding 

- Debugging 

- Test i na 

8. DESIGN OF THE EXPERIMENT 

The experiment was designed to guarantee the most accu- 
rate recording of relevant error information. Therefore/ 
for each of these phases* an appropriate form was designed 
to give a firm au iceline to the experiment programmer for 
recording the entire development of a program including 
necessary details about errors. The error data which were 
recorded on these forms for project 1/ 2/ 3 and 4 are given 

in appendices A, 8/ C and D respectively. For each project 
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the recorded data was analyzed with respect to the following 
f ac t o r s : 

• Number of source statements produced 

- Man hours scent on the project 

- Man hours scent in each of the software 
development chases 

- CPU time used for compiles 

- CPU time used for test and debug runs 

- Number of test and debugging steps neeaed 

• Project time used to correct errors 

• Project time between error detections 

• Occurrence cf errors with respect to software 
development chases 

• Detection of errors with resoect to software 
development phases 

- Time history of error detection and correction 

- Complexity measures of subroutines 

A summary of this analysis is documented at the end of 
each appendix. 

Since it would be too difficult to obtain error data in 
a commercial environments it was neccessary to obtain the 
data by usinq a carefully controlled experiment in an 
acaoemic environment. The considerable overhead necessary 
to record all relevant data was not included in project 
time. It orobaoly would not be feasible to conduct this 
kind of experiment in a commercial environment because: 
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- The overhead woula be very costly. It is unlikely that 
management would support an experiment of this type. 

- Competition among programmers would probably not allow 
accurate results to be obtained. 

- Programmers may feel that the error data would be 
used for performance evaluation. 

In addition to qatherinq Quantitative error data an 
attempt was made to obtain other information about the na- 
ture of errors such as "why was the error made?" or "how was 
the error discovered?". This information could help to dev- 
ise methods for avoiding certain types of errors. For each 
subroutine a directed graph representation was analyzed with 
respect to the following complexity measures: 

- N umber of Nodes 

- Number of Arcs 

- Number of Statements 

- Numbe r of Paths 

- Reachability of Nodes 

- Cyclomatic Number [3<?J 

The results of this analysis are shown together with the 
directed graph representation of the programs in the appen- 
dices. For some structures these measures are not shown 
because the number of paths and the reachability index was 
too large to calculate. 
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PROGRAMMING environment 



The under] y i na program specifications of all projects 
were well defined and not known by the experiment orogrammer 
before the experiment. The experiment programmer was fami- 
liar with the objectives of the experiment and willinq to 
record accurate data to the oest of his knowledge/ even if 
recorded facts seemed to be unfavorable for himself. 

Throughout the experiment software development was done 
in a structured and ohase oriented approach usinq structured 
programming and known software development techniques/ such 
as top-down design/ modularization/ decision tables/ etc.. 

As a orooramminq environment the 0 S / 3 o 0 system was 
chosen as one of the most representative batch processing 
systems available. Batch processing was always used except 
for the final test and debug runs for project # 4 , because 
this program was desianed to run in a time sharing environ- 
ment. During the exoeriment all environmental factors/ such 
as operating system and equioment oeing used and the pro- 
gramming language remained unchanged. Factors related to 
the experiment programmer/ such as ability/ knowledge/ and 
experience did not vary considerably Our ina the experiment. 
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V. QUANTITATIVE RESULTS 



A. DISTRIBUTION OF PROGRAMMING EFFORT 



In Table 1 the distribution of Droject time is shown 
with respect to the major software development phases. 



PROGRAMMING EFFORT (MAN HOURS) 



PROJECT 

PHASE 



Project 

a 1 



Project 
a 2 



Project 
a 3 



Project 

« a 



Total 



DESIGN 



5.0 

(22.9%) 



31.0 
(24. fiX ) 



7.0 

(21.2V.) 



2a. o 

(23.6%) 



67.0 

(23.9%) 



CODING 



7.0 

(32.1%) 



26.0 

( 20 . 6 %) 



a.o 

(12.1%) 



2a. 5 
( 2 a. 3%) 



61.5 

(21.9%) 



DEBUGGING 



a.O 

(16.3%) 



55.0 

( a a . o % ) 



3.0 

( o. IX) 



a 1 .5 

( a i . i % ) 



103.5 

(36.9%) 



TESTING 



SU^ 



5.8 

(26.6%) 

21.8 



13.0 
( 10.4%) 

125.0 



19.0 
(57.6%) 

33.0 



11.0 

(10.9%) 

10 1.0 



46.8 

(17.4%) 

280.8 



TABLE 1 

DISTRIBUTION OF PROJECT TIME DURING THE EXPERIMENT 



B. ORIGIN OF ERRORS VS DETECTION OF ERRORS 



The final statistics of each project contains tables 



showing the numbers of errors made and trie number of errors 



found with resoect to each of the software development 
phases. A summary of these tables is presented in Table 2. 



OF ERRORS 



FOUND 



PROJECT 

PHASE 



Project 
* 1 



Project 
# 2 



P r o j e c t 
a 3 



Project 
tt a 



TOTAL (Percentage) 



DESIGN 

CODING 



1 



( 1.7 % ) 



22 



21 



48 (27.7 %) 



DEBUGGING 

TESTING 

SUM 



1° 



53 



45 

1 

50 



120 (o9.4 %) 



2 ( 1 . 2 %) 

173 (100 % ) 



44 



75 



PROJECT 

PHASE 



OF 



ERRORS MADE 



Project 

a 1 



Project 
« 2 



Project JProject 
a 3 ! a 4 



TOTAL (Percentage) 



DESIGN 


l 

1 


5 


i 

i 


20 


1 

1 


l 

1 


10 


l 

1 


35 


(20.2 %) 


CODING 


1 

1 


38 


i 

t 


53 


1 

1 


4 J 


37 


1 

1 


1 32 


(76.3 %) 


OEBUGb I NO 


l 

1 


1 


i 

i 


2 


1 

i 


i 


3 


l 

1 


6 


( 3.5 %) 


TESTING 


1 

1 




i 

i 




1 

1 


i 




1 

1 


0 


O 

• 

o 

w 


SUM 


1 

1 


44 


i 

i 


75 


1 

1 


4 : 


50 


1 

1 


173 


(100 %) 



TABLE 2 

ERROR DETECTION VS ERROR ORIGIN 
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ERROR CORRECTION TIME AND TIME BETWEEN ERRORS 



1. Error Correction Time 

The mean time to correct an error calculated over 
all 173 errors was 1 2.2 man minutes* 

2* Time Between Error Detections 

The mean time Detween error detections calculated 
over all projects was 6t*6 man minutes. 

D. ERROR TYPES 

All errors which occurred during the exoeriment could be 
identified by one of the oreviously Defined error types. An 
overview of error tvoes with resoect to their frequency of 
occurrence is oresented in Table 3. Error tyoes which did 
not occur are not listed* In addition to the information 
given in Table 3 a bar chart (Table 4 ) is shown which iden- 



tifies 



the most common error tyces 



ERROR 


1 

1 




NUMBER 


OF 


ERRORS 




t 

1 


total 


TYPE 


1 

1 


Project 


! P ro j ec t 


'Project ! 


Project 


1 

1 






1 

1 


n i 


! a 2 


1 

1 


a 3 ! 


u 4 


1 

1 






















D 3 


1 

1 


3 


1 3 


1 

i 


1 

1 


2 


1 

1 


8 


D 7 


1 

1 




1 

1 


1 

l 


1 

1 


1 


1 

1 


1 


DR 


1 

1 


1 


: i o 


1 

l 


l 

l 




1 

1 


1 1 


DIO 


1 

1 




: i 


1 

1 


t 

l 




1 

1 


1 


D 1 1 


• 

1 


1 


: 3 


1 

1 


1 

1 




1 

1 


4 


D12 


1 

1 




: 2 


1 

1 


1 

1 


S 


1 

1 


7 


D 1 3 


1 

1 




i 

i 


« 

i 


1 

l 


1 


1 

1 


1 


D 1 5 


1 

1 




: i 


I 

1 


1 

1 


1 


1 

1 


2 




















C 1 


1 

1 


1 


: i 


l 

1 


l 

1 




1 

1 


2 


ca 


1 

1 


1 


i 

i 


1 

1 


1 

i 




1 

1 


1 


C5 


1 

1 


1 


i 

t 


I 

1 


i 

1 




1 

1 


1 


C6 


1 

1 


1 


i 

i 


1 

» 


i : 


2 


1 

1 


4 


C 7 


1 

1 




i 

i 


1 

1 


i 

i 


1 


1 

1 


1 


C 8 


1 

1 


2 


: i 


1 

1 


i 

i 




( 

1 


3 


CR 


1 

1 




: 2 


1 

1 


i 

i 


1 


1 

1 


3 


C 1 0 


1 

1 


1 


: 2 


1 

I 


i 

i 




1 

1 


3 


Cl 1 


1 

1 


3 


! 2 


I 

1 


i 

t 




1 

1 


5 


C 12 


1 

1 


4 


: 2 


1 

1 


i 

i 


1 


1 

1 


7 


CIS 


1 

1 


1 


« 

i 


1 

1 


i 

i 




1 

1 


1 


C 1 6 


1 

1 




: i 


1 

» 


i 

i 




1 

1 


1 


C 1 7 


1 

1 


1 


: 5 


1 

l 


i 

i 


3 


1 

1 


9 


Cl 9 


1 

1 


1 


i 

i 


1 

1 


i : 




1 

1 


2 


C 20 


1 

• 


2 


: i 


1 

1 


i 

» 




1 

1 


3 


C 2 1 


1 

1 




1 2 


( 

i 


i : 


2 


( 

1 


S 


C 23 


1 

1 


6 


1 5 


1 

I 


i 

i 


1 


1 

1 


12 


C24 


1 

1 


2 


; i 


1 

l 


i : 




1 

1 


4 


C 26 


1 

1 




: i 


1 

» 


i 

i 




1 

1 


1 


C27 


1 

1 


2 


! 3 


1 

i 


i 

i 


4 


1 

1 


9 


C 28 


1 

1 


5 


! b 


1 

i 


i 

i 


6 


1 

1 


1 7 


C 29 


1 

1 




: 2 


1 

l 


i 

i 




1 

1 


2 


C 30 


1 

1 




: i 


( 

1 


i 

i 




1 

1 


1 




















A 1 


« 

1 


2 


: i4 


• 

i 


i 

i 


1 6 


1 

1 


32 


A 2 


1 

1 


2 


: i 


1 

1 


i 

i 


1 


1 

1 


4 


A3 


1 

1 




i 

% 


1 

1 


i 

i 


1 


1 

1 


1 




















B 3 


1 

1 


1 


i 

i 


1 

1 


i 

« 




t 

1 


1 


B4 


I 

1 




: 2 


1 

1 


i 

i 


1 


1 

1 


3 



TABLE 3 



ERROR DISCOVERY WITH RESPECT TO ERROR TY D ES 



ERROR 


1 

1 




TYPE 


1 

1 


FREQUENCY OF OCCURRENCE 








03 


1 * 


* * * * * * * 


09 


I * 


********** 


Dll 


! * 


* * * 


D12 


» * 


****** 


C6 


! * 


if if if 


C 1 1 


! * 


* * * * 


C 1 2 


! * 


****** 


C 1 7 


» * 


******** 


C21 


I * 


* * * * 


C23 


5 * 


*********** 


C2a 


1 * 


* * * 


C27 


j * 


******** 


C28 


! * 


**************** 


A 1 


! * 


**************************** 


A2 


| * 


* * * 


Other 


i 

i 




Errors 


! * 


************** ************** 



TABLE a 

MOST FREQUENT ERROR TYPES 
\ 

E. ERROR OCCURRENCE RELATED TO COMPLEXITY MEASURES 

One of the objectives of the experiment was to determine 
the re 1 a t i ons h i o r if any, between error occurrence and the 
structural properties of subroutines. Of particular in- 
terest was the analysis of structural properties with 
resoect to the error simulation work done by Sc hne i dew i nd , 
Howard and Ki rchaaessner (331. In addition the cyclomatic 
number as defined by McCabe (32] for software engineering 
purposes, was used in the analysis. The results of this 
analysis are shown in Table 5. The follow inq definitions 
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and terms are used in Table 5: 

N p Number of Paths (minimum number of paths: no 

Iood traversed more than once in succession) 

V Cyclomatic Number: number of independent circuits = 

number of arcs - number of nodes + P 

R Reachability: summation, over the nodes, of number 

of ways of reaching a node 

r Average reachability: R/number of nodes 

S Number of Source Statements 

e Number of Errors Found in Actual Program 

Tf Labor Time Required to Find Errors 
(Since Previous Error Detection) 

Tc Laoor Time Required to Correct Error 

All of the above are with respect to a single program. 
The definition of V includes an implicit arc connectinq the 



start and terminal noces (strongly connected graph) 



TABLE 5 



COMPLEXITY MEASURES VS. ERROR PROPERTIES 
Part A - Procedures With One or M 0 re Errors 

COMPLEXITY MEASURES ERROR PROPERTIES 



Elf Etc 



Project/ 

P roc edu r e 


No 


V 


R 


r 


S 


e 


C M a n - 
mi n s 1 


(Man- 
mi n s ) 


------- — 


— — 


— 


— 


— 


- 


- 


----- 


- — - - 


1/1 


2 


2 


7 


l .a 


1 9 


1 


35 


1 0 


1/5 


* 


o 


* 


* 


2b 


5 


95 


53 


I/O 


★ 


5 


* 


* 


1 


2 


37 


35 


i /a 


★ 


5 


* 


* 


21 


1 


35 


15 


1/9 


2 


2 


8 


1 .333 


fa 


1 


1 15 


20 


2/1.19 


1 


1 


2 


1 .0 


3 


1 


1 0 


10 


2/1.23 


1 


1 


2 


1 .0 


1 1 


1 


1 1 0 


10 


2/2.2 


2 


1 


a 


1 Jo 


8 


1 


15 


10 


2/7 


3 


2 


7 


l . a 


15 


3 


230 


95 


2/9 


72 


8 


370 


19.079 


95 


3 


190 


185 


2/10 


9 


0 


25 


2.778 


1 8 


1 


10 


5 


2/1 1 


★ 


fa 


* 


* 


59 


3 


950 


65 


2/12 


5 


2 


1 3 


l . a a a 


39 


2 


300 


3 0 


2/15 


★ 


a 


* 


* 


19 


1 


5 


1 


2/ 1 o 


★ 


5 


* 


* 


30 


2 


150 


20 


2/ 1 e 


12 


a 


50 


2.99 1 


2b 


1 


5 


15 


2/21 


* 


l fa 


* 


* 


99 


8 


750 


195 


3/3 


2 


2 


6 


1 .5 


13 


1 


fa 0 


5 


* Very 1 


arqe va 


i ue 















TABLE 5 
(continued) 



Part A 


- Procedures 


With One 


• o r 


More 


E r ro r s 




Project/ 
Proceou re 


No 


V 


R 


r 


S 


e 


ETf 

(Man* 

mins) 


Yj c 

(Man- 

mins 


......... 


- - 


- 


- 


- 


— 


— 


----- 


- — — — 


a/7 


40 


6 


151 


3.a38 


83 


1 


120 


5 


a/ 1 3 


1 6 


5 


6a 


a .267 


28 


1 


20 


15 


a/ 1 a 


★ 


8 


★ 


* 


37 


5 


255 


o5 


a/is 


4 


3 


12 


2.0 


13 


2 


ao 


35 


a/2i . 3 


7 


3 


3a 


a. 857 


lo 


1 


0 


5 


a/22 


* 


7 


* 


* 


3a 


1 


lo0 


30 


a/23 


18 


5 


60 


a . 6 1 5 


2a 


1 


125 


10 


a/27 


5 


a 


2 3 


2.091 


1 8 


3 


360 


120 


a/28 


* 


5 


* 


* 


35 


2 


90 


50 


a/29 


321 


1 3 


l a68 


5a. 370 


a9 


5 


1 125 


160 


a/30 


6 


a 


2a 


2. a 


19 


1 


60 


30 


a/3i 


* 


a 


* 


* 


27 


1 


30 


10 


a/33 


14 


a 


lb 


7.0 


1 7 


2 


135 


1 0 



* Very large value 
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FABLE 5 
(continued) 



Part B - ConDlexitv Measures for Procedures 





with 


Zero 


Errors 






Project/ 


Procedure 


Nd 


V 


R 


r 


s 


--------- 


- - 


• 


- 


- 


- 


1/2 


3 


2 


8 


1.333 


6 


1/3.1 


1 


1 


2 


1 .0 


8 


1/3.2 


1 


1 


2 


1 .0 


1 1 


1/3.3 


1 


1 


2 


1 .0 


a 


1 /a 


o 


3 


26 


a. 333 


1 8 


1/7 


7 


3 


ao 


5.0 


15 


2/1.1 


1 


1 


2 


1 .0 


3 


2/1.2 


1 


1 


2 


1.0 


3 


2/1.3 


1 


1 


2 


1 . 0 


3 


2/1 .a 


1 


1 


2 


1 .0 


3 


2/1.5 


1 


1 


2 


1 .0 


3 


2/1 .6 


1 


1 


2 


1 .0 


3 


2/1.7 


1 


1 


2 


1.0 


3 


2/1.8 


1 


1 


2 


1 . 0 


3 


2/1 .9 


1 


1 


2 


1 .0 


5 


2/1.10 


1 


1 


2 


1 .0 


5 


2/1.11 


1 


1 


2 


1 . 0 


5 


2/1.12 


1 


1 


2 


1 . 0 


1 3 


2/1.13 


1 


1 


2 


1 .0 


3 


2/1 . 14 


1 


1 


2 


1 .0 


3 


2/1.15 


1 


1 


2 


1 .0 


3 


2/1 . 16 


1 


1 


2 


1 .0 


3 


2/1.17 


1 


1 


2 


1 .0 


3 


2/1 . 18 


1 


1 


2 


1 .0 


3 


2/1 .20 


1 


1 


2 


1 . 0 


3 


9/1.1 


1 


1 


2 


1 . 0 


2 


9/1.2 


1 


1 


2 


1 .0 


2 


a/i.3 


1 


1 


2 


1 .0 


7 


a/i .a 


1 


1 


2 


1 .0 


5 


a/1.5 


1 


1 


2 


1 .0 


7 


a/i .6 


1 


1 


2 


1 .0 


5 


a/i.7 


1 


1 


2 


1 . 0 


5 


a/i.8 


1 


1 


2 


1 . 0 


5 


a/ i .9 


1 


1 


2 


1 .0 


5 


a / 1 ,io 


1 


1 


2 


1 .0 


a 


a / 1 . 1 1 


1 


1 


2 


1 . 0 


3 


a/1.12 


1 


1 


2 


1 .0 


3 


a/1.13 


1 


1 


2 


1 . 0 


3 


a / i . la 


1 


1 


2 


1 .0 


3 


a/1.15 


1 


1 


2 


1 . 0 


3 


a/ 1 . 1 6 


1 


1 


2 


1 .0 


3 


a/i . l 7 


1 


1 


2 


1 .0 


3 



U7 



TABLE 5 
(continued) 



Part B - Complexity Measures for Proceoures 





With 


Zero 


E r r o r s 






Project/ 


Procedure 


No 


V 


R 


r 


S 


— 


- - 


- 


- 


- 


- 


4/1.18 


1 


1 


2 


1 . 0 


3 


a/i . IP 


1 


1 


2 


1 .0 


5 


a/ 1.20 


1 


1 


2 


1 .0 


5 


a/i.2i 


1 


1 


2 


1 .0 


6 


a/i .22 


1 


1 


2 


1 . 0 


9 


a/i.23 


1 


1 


2 


1 . 0 


6 


a/2.1 


2 


1 


a 


1 .0 


8 


a/2.2 


2 


1 


a 


1 .0 


9 


a/2.3 


2 


1 


a 


1 . 0 


9 


a/3 


2 


2 


6 


1 .5 


a 


a/a . l 


2 


2 


8 


1 .6 


7 


a/a .2 


2 


2 


8 


1 .6 


Q 


a/5 


8 


a 


38 


a. 222 


5b 


a/6 


a 


l 


6 


1 .0 


2a 


a/e.l 


2 


2 


8 


1 .6 


1 3 


a/8.2 


2 


2 


8 


1 .6 


1 3 


a/8.3 


2 


2 


8 


1 .6 


1 0 


a/8. a 


2 


2 


8 


1 .6 


9 


a/8.5 


2 


2 


8 


1 .6 


12 


a/9 


1 6 


5 


95 


7.917 


21 


a/ l o 


12 


5 


65 


a. 6a3 


99 


a/ 1 1 


7 


3 


38 


5.929 


19 


a/12 


* 


a 


♦ 


* 


20 


a/i6.i 


2 


2 


6 


1 .5 


6 


a/ 16.2 


2 


2 


6 


1.5 


12 


a/16.3 


2 


2 


6 


1.5 


9 


a/ 16 . a 


2 


2 


6 


1 .5 


1 0 


a/ i 7 


1 6 


1 


18 


1 .0 


21 


a/i8 


8 


a 


a2 


3.818 


21 


a/ i 9 


a 


3 


16 


2.667 


1 1 


a/20 


3 


2 


9 


1.125 


1 3 


a / 2 1 . i 


7 


3 


3a 


a .857 


ia 


a/2a 


16 


7 


83 


a .368 


19 


a/25.i 


2 


2 


8 


1.333 


15 


a/25.2 


2 


2 


8 


1.333 


1 0 


a/25.3 


2 


2 


8 


1.333 


1 7 


a/26 


3 


3 


15 


1 .5 


1 9 


a/32 


7 


3 


36 


5.193 


15 


a/3a 


2 


2 


5 


1.25 


15 



* Very large value 



Using the results of Part A in Table 5, the following 



correlation coefficients were calculated: 



V 


vs* 


e : 




. 7834 


s 


vs • 


e : 




.5880 


V 


v s • 


2 T f 


• 


.6734 


V 


v s . 


2Tc 


• 

• 


.7220 


s 


v s • 


ZTf 


• 

• 


.5902 


s 


vs . 


2Tc 


• 

• 


.5091 


V 


vs . 


S : 




. 7903 



The relationshio between mean complexity values and 
error occurrence is shown in Table 6. 



Zero Errors One Or More Errors 



81 


Proceaures 


3 1 P rocedures 


Mean Cvclomatic 
Numbe r ( V ) 


1 .68 


4 .74 


Mean Number of Source 


Statements (S) 


9.33 


27.23 



TABLE 6 

RELATIONSHIP BETWEEN MEAN COMPLEXITY 



VALUES AND ERRORS FOUND 



VI 



SUMMARY AND CONCLUSIONS 



A. GENERAL REMARKS 

Due to limited man power and time constraints the amount 
of test data was too small to allow major conclusions. 
Furthermore the scope of the experiment did not include some 
important programming problems such as real time and 
mathematical oroblems. However, taking all constraints into 
consideration some Qualitative aspects were recognized which 
could be important factors in subsequent studies. 

9. SIGNIFICANCE OF ERROR TYPES 

Although the cateaor i zat i on and definition of error 
tyoes was not considered complete/ all errors detected in 
the experiment could be identified using these error defini- 
tions. The most frequent error type M A 1 " (Manual Error) 
seems to be related to the number of source statements 
whereas the frequencies of the remaining errors seem to oe 
related to the complexity of the programming problems. 

The recording of errors with resoect to their tyoes con- 
tributes to a learning process which enables the programmer 
to reduce the number of errors made on subsequent projects. 
It was felt that this concept could also be effectively used 
in a commercial software production environment. The error 
recording would have to be private to the individual 
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programmer and used by the programmer alone* Otherwise it 
would have a negative influence on working atmosphere and 
performance. If the evaluation process were assisted by 
appropriate software packages/ the overhead in analyzing 
errors would be reduced to a great extent. The overhead of 
recording the errors for the individual programmer tends to 
decrease after the recording svstem has been learned. 



C. COMPLEXITY MEASURES 



Several measures of complexity calculated for each 
subroutine are presented in the aopendenc i s together with 
subroutine directed graph representation. when the number 
of errors found in procedures was correlated with cyclomat- 
ic number and number of source statements/ the correlation 
coefficients were higher than for other complexity measures 
( 3 4 J . It also aooeared that these two measures were related 
to the total error detection and total error correction 
times. It was learned that trying to keep the cvclomatic 
number small not only reduced the numoer of errors but also 
contributed to the reduction of debugging and testing ef- 

'i 



fort. This experience supports the results of McCabe's 
analysis of structural properties of programs with respect 
to difficulties in testing and debugging [3£]. It should be 
noted that the influence of other factors such as usage of 
structured programming techniques and working habits of the 
experiment programmer also contributed to these results. 
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D. SOFTWARE ENGINEERING ASPECTS 

Throughout the experiment comments were made to indicate 
possiole clues about error sources or ways to avoid certain 
types of errors. An evaluation of these comments showed 
that some of these errors would have been avoided by the 
usage of decision tables or aoolying a proper desk test. 
Many key punch errors of type ” A 1 ” were not recognized dur- 
ing the process of ounchinq carcis, because the design of the 
key punch machines (IBM 2 Q ) does not provide an immediate 
control of characters being punched. It is desirable to see 
the result of each key stroke instantaneously. The freauen- 
cy of error occurrence of type 11 Manual Error” in former pro- 
jects done by the same programmer using console editing was 
considerably lower than the error frequency discovered in 
this experiment. 

Software engineering concepts such as too-down design, 
design review and Structured Walk Through were used 

throughout all projects. These methods had a major influ- 

\ 

ence on the software development process and were considered 
to be extremely useful. 

The usage of ALGOL w had a major influence on the pro- 
gramming style. Variable names were chosen to make Programs 
self-documenting to a great extent. Therefore most program 
parts were easy to read and to understand which implied a 
reduction of the debugaing effort. 
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Another concept which was successfully employed was the 



idea of using identifiers which differ by two or more char- 
acters . The idea of implementing some kind of "Hamming Dis- 
tance" [353 between identifiers was derived from bitter 
experience in former projects and heloed to avoid errors 
which occur * h e n a single error of tyoe "Manual Error" or 
"Mental Error" produces a different identifier which matches 
with one of the previously defined identifiers. Thus this 
kind of error did not occur during the compiling process* 

E. CONCLUSIONS 

Errors in software projects appear to be related to pro- 
gram structure and to the number of source statements* 
Although this experiment was not 1 arae enough to state these 
relations in a formal manner, the results were encouraging, 
Cyclomatic numper and number of source statements could 
prove useful as guidelines for managing proaramming pro- 
jects* 

The usage of too-down design^ structured programming 
techniques and other software development tools should be 
mandatory for program mina projects. The recording and 
evaluation of software errors would be a reasonable approach 
to avoid errors in programming projects and to reduce cost. 

Detailed debugging on a modular basis and development of 
large programs by stecwise intearat ion of thoroughly de- 
bugged modules seems to be a good strategy to achieve 
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reliable programs 



F. RECOMMENDATIONS 

Similar experiments would reveal more of the relations 
between software errors and complexity of programs. More 
definite information about the utility of complexity meas- 
ures could be obtained by using these measures on large- 
scale software projects in a commercial production environ- 
ment. 
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APPENDIX A 



PROJECT DESCRIPTION 

Project # : 1 

Project title : PALINDROMES 
Programmer : HOFFMANN 
Programming Language : ALGOL 

Programming environment: I8M/360/67,OS/MVT>8ATCH 

Design notes : see ANNEX A 

Program listing : see ANNEX B 

Coding notes : see ANNEX C 

Debugging notes : see ANNEX D 

Error Listing : see ANNEX E 

Final statistics : see ANNEX F 

Graphical representation : see ANNEX G 

Test phase description: see ANNEX H 

Starting date: 28 JAN 77 Ending date: 6 

EXPERIMENT DESCRIPTION 
1. Project description: 

FIND PALI NDROMS 

A palindrome is defined as a character string of 
n, which has the following characteristics: 

character [ i ] = character tn - i - 11 for al 

i=0 to i=n/2 (integer division) and n >= 



FEB 77 



1 eng t h 

1 i from 

2 
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Given an input string of length n , where 2 <= n <= 25b, 
find all occurrences of Palindromes. 

The input is given in form of punched cards. 

The length of the string ( " 1 " ) preceeas the input of the 
character string. 

All information being read should be printed. 

Give as a result the number of palindromes of each length 
and the position of the starting character of each palin- 
drome in card number and character position. 

Find only maximal length palindromes, (i.e. "ABC C8A" should 
be recorded as the palindrome of maximal length, whereas the 
included palindromes "BC C B " and "C C M should be exclud- 
ed.) 



2. Programmer's background: 

a) Experience in proaramming: 



Oc t 


1970 - May 


1971 


Proaramming courses 


May 


1971 - April 


1972 


Module Programmer 


May 


1972 - June 


197a 


work in Test and Simulation Depart- 



ment at the 

naval COMMAND AND CONTROL systems 
COMMAND (FEDERAL GERMAN NAVY) 

Testina of tactical real time systems 
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March 1975 - Jan 1977 Student at the NAVAL POSTGRADUATE 

SCHOOL, Monterey, Computer Science 

b ) Experience in testing: 

Two years of work in testing and simulation, 

c) Experience in the area of the given problem: None, 

d) Experience in the progamminq language being used: 
Exoerience over a period of 18 months in more than 10 

orogramming projects. (Total number of source statements 
produced durina that time was more 4000,) 

3. Psychological factors: 

a) Did the programmer like the project? Yes. 

b) how aoes the programmer like the Programming 
1 anguage? 

Favorite programming language. 

c) Was the programmer satisfied by the way the o r o D 1 e m 
was specified? 

Only minor criticism. 

d) How did the programmer like the programming environ- 
ment? 

The facilities (study room, card punch room) were not 
conducive to efficient programming because of restricted 
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space, bad lighting and noise. 
e) Other factors: 

The recording of the exoeriment's data during the 
ject affected speed and concentration considerably. 



pro- 
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4. Comments on Documentation 

For the documentation of each software development phase a 
special documentation form has been developed. These forms 
are designed to provide a f i rt*\ gu i de 1 i ne for the experiment 
programmer to record all data of interest for subsequent 
error analysis, 

- Beain and end of each step was recorded with respect to 
day and time, 

• Each error was recorded when it is discovered. The error 
was then identified by a unique error number (1/2/..,). 
Furthermore the time of discovery and the error tyoe (using 
error types listed in ANNEX F) were recorded. 

- If appropriate/ comments about error discovery/ reason why 
the error was made/ etc. were documented in ANNEX E2. 

- For each error the phase in which the error was made/ the 
phase in which the error was discovered and the time spent 
to correct the error was recorded in ANNEX El. 

• For each step in any one of the software development 
phases the day/time of begin and end was recorded. In addi- 
tion/ the time (in man hours) for each step was recorded. 
This excludes the overhead used for documentation of the 
experiment data. 
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ANNEX A 



Page 1 



of « 



WORKSHEET for oesign PHASE and design review phase of project # 1 



STEP 

* 


PROBLEM ANO PLANNEO SOLUTION 


alternate 

SOLUTIONS 


! 

DAY 

TIME 


MAN 

HOURS 

/STEP 


ERRORJ COMMENT 
n J 


1 


Too*0own Design: 




1/28 




t 

« 

« 

« 




Use the following buildina 




1 0 a o 




1 




blocks: 








i 

i 




-Initial i z a t i o n 








i 

i 




-Reaa and Write Input Cards 








i 

• 




-Find all Palindromes which 






1.5 


• 

• 




are not totally included in 








i 

• 




a 1 arge r oa 1 i ngrome 


-Oo not allow 






i 

• 




rind also palindromes 


o ve r 1 aop i nq 






i 

• 




which are overlapping: 


na 1 i ndr ome s . 






i 

i 




A3B Ax A will be recorded as 


This allows two 






i 

i 




two palindromes: 


solutions: 






i 

i 




ABBA and A X A 


1 • Always record 






i 

i 




-write all palindromes beina 


the first palin- 






« 

• 




found 


drome • 






« 

i 




-The necessary inputs 


2 . A ) ways record 






i 

« 




1. length of string 


the la rge r oa- 






• 

• 




2. character string of given 


1 i nd r ome • If oa- 






i 

* 




1 eng t h 


1 i ndromes are of 






i 

• 




are expected from cards. 


equal length# 






« 

* 






then record the 


1/2Q 




i 

« 






first one. 


1330 




i 

i 

i 


a 


Specify I n i t i a 1 i z a t i on : 




1/28 




i 

i 

t 




-Initialize all arrays and 




1 330 




i 

« 




variables 








« 

i 




-Write explanatory text 








i 

i 




-Read lenqth of input strinq 








i 

i 




and check (assert 2< = lenqth 








i 

i 




of St r i nq <= 256 ) 








i 

i 




-Set integer field size to 5 








* 

i 




-Use all 80 columns of the 






.5 


1 




input cards (specify as com- 


C a r d 1 i mi t ccul d 






• 

• 




pile parameter CARO LIMIT) 


be soec i f i ed by 






• 

• 






user via incut 


1/28 




i 

i 






parame ter. 


lao o 




• 

• 



Remarks : 

Because of the small size of the Project the system design phase 
is omitted. 



Total man hours soent in design: 5.0 
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ANNEX A 



Page 2 of 4 



WORKSHEET FOR DESIGN PHASE AND DESIGN REVIEW PHASE OF PROJECT u 



MAN 



! PROetEM AND PLANNED SOLUTION 


alternate : 


{DAY ; 


! HOUPS 


ERROR,' 


1 

t 


SOLUTIONS ! 


! TIMEI/STEP 


# : 


! Soecify Reading ana Writing 




1/28 






! o f Inpu t Ca r ds : 

{-Number of input cards is de- 
pendent on CARDLIMIT and soe- 
! c i f i e d length of character 

{String. 

lit is calculated: 

{Number of incut cards= 
{(length of string -1 )/card- 




1 400 






! limit + I 

{-Maintain a card counter 
{-Read and write each card ore- 
{ceeded bv the appropriate 
{ c a r d numbe r . 




1/28 

1500 


.5 




I Data Design: 




1/28 






{-Use a text buffer (Text) 




1500 






(String Array Indexing from 1 
! t h rough 256 ) 




1505 




1 ! 


{-Use an I/O bu f f e r ( C a r dbu f f e r ) 
{-Use two integer arrays to re- 
Icord begin and end positions 
Jof all palindromes. 






.5 




{ ( 5eg i n-o f -pa 1 i drome r End-o f - 

{ Dal indrome) 










{-Use other seif documenting 










! common data: 

{ (Carol i mi t# Length -of -text, 

{ Bu f f e roos i t i on > Cardcounter, 
{ Pal indromecounter) 










{-Use Index variables IX, JX 
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{-Use Local variables as needec 




1530 







will be 
redes i gnea 
curing 
design 
r e v i e v# ( D 3 ) 



Remarks: 

Error #1 was made during step 2, 



ANNEX a 



Page 3 of 4 



rtORKShEET FOR DESIGN PHASE AND DESIGN REVIEW PHASE OF PROJECT * 1 











; mam 


i 




STEP 


problem and planned solution 


ALTERNATE 


DAY 


'.HOURS 


J ERROR,' COMMENT 


u 




SOLUTIONS 


T I'-'E ! /STEP 


! * 


i 

• 


S 


F i nd Pali ndromes : 




1/28 










-Scan text from left ro riqht. 


-Sc an text f rom 


1530 










If palindromes of minimal 


left to right. 












length ( i . e . length is 2 or 3) 


Consider every 












are found# call a subroutine 


character as the 












which checks whether or not 


oo s s i b 1 e oeq i n- 












the oal indrome is the center 


ninq of a cal i n- 




.3 








of a larger oal marome. 


drome . 

Check for all 
ooss i b 1 e 1 engt hs 
going backward 
f rom the end of 
the text string 
down to a s t r i ng 


1 /28 












1 enqt h of 2 . 


1550 








0 


Recorai ng of Pal maro^es: 




1 /28 










-Record a oalindrome only if 
it is not entirely included in 
any of the previously recorded 
ca 1 i ndromes . 

-It is also necessary to check 
whether any of the oreviously 
recorded oal indromes is en- 
tirely included in the palin- 
drome last detected* 




1550 


.5 








-A flag should be set to make 
sure that only maximum 1 enqt h 




1/28 










oal inaromes *ill be orinted* 




1 o20 








7 


write all Palindromes: 




1/28 










-Identify the oosition of each 
palindrome by begin and end 
with resoect to inout card 
number and position on the 
card. 




1 o20 










-Using the position in the 
text string and the oarameter 
CARDLIHIT the input card num- 
ber and the desired character 
oositions can be calculated. 




1/28 

1630 


.2 







Remark s : 
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annex a 



Page a of a 



WORKSHEET FOR DESIGN PHASE AND DESIGN REVIEW PHASE OF PROJECT * 1 



( 






man ! 




STEP! PROBLEM AND PLANNED SOLUTION 


alternate 


DAY 


HOURS j 


ERROR! COMMENT 


u 




SOLUTIONS 


TIME 


/STEP! 


; » i 


8 


Design Review! 




1/28 




• 

• 

• 

• 




-Input cards will be stored 




2100 




! Forgo tten 




into TEXT(string array) con- 








! action in 




sidering CARDLIMIT. 








! step 2 

i 




-Preceeding the writing of the 




2115 




i 

2 ! Forgotten 




recorded oal indromes a prooer 








I action in 




headline should be orinted. 








! steo 6 ( D3 ) 




(TEXT2) 








i 

i 

* 




-If no palindromes have been 








• 

i 

t 




found/ write eaui va 1 ent mes- 








i 

i 




sage, (TEXT3) 






i ,0 


i 

i 

• 




-Design of printout format. 




1/28 




• 

! See remar k s 








2200 




i 

i 



Rema r < s : 

Design of Printout For r, at: 

Palindrome 9egin End 

Number Card Character Cara Character 
Number Position Number Position 



n bnc bcp ecn ecp 

<oa 1 i ndrome > 



.1 h e r e 



n is the seauence number 
c a r a numbe r of 
character oosi 
card number of 
charcter oosit 



DCn 


i s 


the 


bcp 


i s 


the 


ecn 


i s 


the 


ecp 


i s 


the 



oalindrome fceain 
t i on of oalindrome begin 
oal i ndrome end 
ion of oalindrome end. 
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Annex B 



Program Listing of Project # 1 



6a 



X 

O 





X 




«» 




1— 












15 




CO 




X 
















o 




X 












CO 


CL 


CL 




1— 










o 


UJ 


UJ 


<t 














X 


X 




O QC 


• #> 


UJ 










►-H 


a 


O 


X 


m 


h- 










C£ 


CL 


< 


q£ • *>1— 


X 












f— 


a 


CL 


O-^X 


X 


X 




• *» 






CO 


X 


< 


x>ox 


X 


3 : 


X 


— * 




X 




h—t 


X 


mu 


< 




X 


K- 




o 


CL 


X •• 


o 


Q1CMO 


HH 


•* 


X 


X 




X 


UJ 


<IUJ 




X •• | 


X 


►— 


h- 


X 




X 


H- 


Q.S 


CO 


LL •• Q 


<L 


X 




H- 




o 


O 


a 


X 


•-X— ic£ 


> 


X 


— 


1 






< 


xa: 


►— 4 l 


0D^< 




h- 


— • 


X 




> 


CL 


coa 


cxcoxo 


X 


1 


^0 


LJ 




f— 


< 


XX 


i— i 


x 21 


UJ 


X 


m 


h- 




•-LU 


X 


1H 


zcxuu •* 


X 


X 


CXI 


xi- 




OCX 


o 


h™ X 


oi-^acx 


X 


1 




h-l-H 




xc 




< 


OCO^OCJ 


1— 1 


X 


A 


ox 




!-</> 


< 


>CL 




X—* 




1— 




xx 




X 




X 


h- 


l-HHh- 


H- 


o 


1— 


LU 




OH 


X 


-Z.CL 


X 


^ x»— » 


X 


X 


X 


XX 




ox 


HH 


OX 


X 


X<CO 


X 


X 


X 


*< 




ox 


X 


o 


X 


XCLO 


s: 


X 


h- 


: X 




X 


h- 


lUcL 


X 


X 1 Cl 


X 




1 


QC 


• #* 


x^: 


►— 4 


o< 


a 


OX0C 


u 


o 


X 


••O 


— * 


<o 


jSL 


X 


o 


uux 


o* 


<x 


u 


C02 


X 


oo 




•h- 




M- 




X 


1 


m aj 




o 


CO 


Q CO <T 




ox 




OC 


X 


<x 


X 


X 


LU 


UJhh 




xo 






1- 


o 


o 




X 


hJZ 




xco 




•» 


o 


ZK 


X 


'r- m 


a 


CO ►"« 








CO 


X 


HZ 


< 


X 


QC 


*-»< 




•> •• 




X 


X 


c£X 


> 


X 


a 


x Q 




XH 




X 


X 


1-21 




X 


x • 


LUX 




xx 




CO 




CO 21 


ac 


X 


M sQ 


XOQ 




OX 




< 




o 


UJ *•* 


o 


xi n 


CQXX 




CLh- 




h-l 


QC 


1-0 


OcO 


o 


<C\J 


OX 




X | 




GC 


o 


*•3 


XX 




Cl 


♦xoo 


— 


XX 




< 




= X 


l-X 




II CUXOCZI 


nO 


mQ •• 




> 


— * 


••X 


x*— 




CO 
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annex c 



Page 1 of 1 



WORKSHEET FOR COOING PHASE OF PROJECT * : 1 
Beginning of Coding (day/time) * 1/28/2200 

End of Coding (day/time) * 1/31/1220 
Man hours : 7.0 (including punching of cards) 



1 ) 



COOING 

BEGIN ! END 

DAr/TI^E ! DAY/TtME 


PROGRAM PART 


EROOR ; 0 A Y 

* {TIME 

9 

t 


comment 

(incl. coded error types) 


• 

1 








1) Record when error is 


1 

• 








detected. 


1/28/2200 1 


-Data Definition 








i 

• 


-Initial ization 








• 

• 




3 


2205 


C23 


• 

• 




a 


2210 


C2 3 


! 1/28/2300 

1 










1/30/1335 ! 


-Utilities 








i 

i 


(texts# I/O 


5 


1 3ao 


CIO 


i 

■ 


sue rou tines) 


6 


laoo 


C 8 


i 

i 




7 


1 a05 


C 6 


i 

i 




8 


1430 


C23 


i 

i 




9 


1500 


C23 


i 

■ 




10 


1510 


03 


1 1 /30/1530 

i 










1/30/1615 ! 


-Pali no rone 








• 

• 


chectc 


1 1 


1630 


Cl Misunderstanding 


t 

• 








of design 


: 1 /30/ 1 700 
• 










1/30/1700 ! 


Coding Review 


12 


1710 


Cl 1 


1 

1 




13 


1710 


Cll (wrong indentation) 


1 

1 




la 


1 720 


C 1 7 


• 

1 




15 


1 725 


C27 


! 1/30/1730 
• 










1/31/0930 ! 


Pune h i ng Cards 


1 6 


1030 


C 1 5 


1 

1 




17 


1050 


C 4 


1 

1 




18 


1110 


C20 


1 

1 




19 


1115 


C28 


1 

( 




20 


1 125 


C 1 2 


t 

» 




21 


1 125 


C1P 


1 

1 




22 


1 1 30 


C24 


• 

1 




23 


1135 


C 1 2 


( 

t 




2 a 


1 140 


C 2 4 


; 1/31/1220 











Renfia r k s : 

Man hours scent for ounching cards: 2.8 



70 



ANNEX 0 



Page 1 of 4 



WORKSHEET FOR DEBUGGING PHASE 
PROJECT # : 1 DEBUG Run U : 1 

Begin of Debug Run (day/time) : 1/31/1334 

End of Debug Run (day/time) : 2/04/1200 

* of Debug Steos incl. in Debug Run: 3 CPU time for Debug run (sec) 

CPU time for necessary comoiles (sec) : 5,31 
a) 1.69 b) 1.49 C ) 2.13 d) e) f) g) 

Man hours for this DeOuq 3 u n : .9 (excluding overhead) 



,02 



STEP I PROGRAM 
a { PART 



OBJECTIVE AND EXPECTED 
RESULT 



ACTUAL 

RESULT 



MAN l : 

DAY ; HOURS ; ERROR 
T IME ! /STEP J a 

i i 


) 

COMMENTS 
AND COOED 
ERROR TYPES 


1/31 


.3 




1 ) Record when 
error 
occurs 


1 33S 




25 


A 1 






26 


C5 






27 


C 1 2 






28 


A2 






29 


C23 






30 


C 1 2 






31 


Cl 1 


1/31 

1400 




32 


A 1 


1/31 


.2 


33 


C23 


1806 

1/31 

1830 

2/04 


.4 


34 


A2 


1121 

2/04 

1200 




35 


C28 



A 1 1 
oar t s 



A 1 1 
03 r t s 



A 1 1 
carts 



Get errorfree comoile 



8 como i 1 e 
e r ro r s 



Get errorfree comoile 



Get errorfree comoile 



2 c omo i 1 e 
e r r o r s 



0.K . 
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ANNEX 0 



Page 2 of 4 



WORKSHEET for DEBUGGING PHASE 
PROJECT * : 1 DEBUG Run * : 2 

Begin of Debuq Run (day/time) : 2/05/1330 

End of Debug Run (day/time) : 2/05/1500 

U of Debug Steps i nc 1 . in Debug Run: 6 CPU tine for Debug run (seel: .08 

CPU tine for necessary compiles (sec) : 4.99 



a) 2.11 b) 1.46 c) 1.42 d) 


e) 




f ) 


a ) 


Man hours for this Debua Run : 1*7 (excluding overhead) 














vi A N 


1 ) 


STEP J PROGRAM OBJECTIVE AN0 EXPECTED 


ACTUAL 


DAY .'HOURS 


1 ERROR 


COMMENTS 


U ! PART 


RESULT 


RESULT 


TIME 


! /STEP 


! 3 


ANO CODED 


■ 








1 

1 




ERROR TrPES 


i 












1 ) Record when 


i 

i 












error 


i 

• 












occurs 


I ! U t i 1 i - 


Test all Texts exec t 


Pun 


2/05 


• 6 


36 


01 1 


! ties 


TEXT3 (all texts should 


error 


13^5 








1 

1 


be crinted as designed ) 












1 

1 






2/05 








1 

1 

1 






1420 








2 : Ut i 1 i- 


Reoeat steo 1 


Run 


2/05 


.5 


37 


C 8 


J ties 




error 


1420 




38 


C27 


i 

i 










39 


C20 


i 

• 






2/05 








i 

• 

• 






1445 








3 ! Ut i 1 i - 


Repeat steo 1 


Spac i nq 


2/05 


. 6 






J ties 




p r ob 1 en s 


1445 




40 


C 2 8 


4 J Ut i 1 i - 


Test spacing and printing 












! ties 


of bl an x lines 


O.K . 










i 

i 


(all formats should ap- 












i 

« 


pear as aesiqned) 












5 ! Ut i 1 i - 


Test calculation of num- 












! ties 


ber of input cards for 


O.K . 










1 

1 


m innun length of input 












1 

1 


string (2 characters) 












1 

• 


(program should read on- 












• 

1 


1 y one card) 












6 J U t i 1 i - 


Test detection ana writ- 












! ties 


ing of a palindrome of 












i 

i 


mini nun length 


O.K. 










« 

• 


(palindrome should be de- 












• 

• 


tected ana listea as de- 












• 

• 


si gnea ) 












• 

i 






2/05 








i 

i 






1500 
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annex d 



Page 3 of 4 



WORKSHEET FOR DEBUGGING PHASE 
PROJECT n : 1 DEBUG Run * : 3 

Begin of Debug Run (day/time) : 2/05/1500 

End of Debug Run (day/tine) : 2/05/1600 

u of Debug Steps i no ) . in Debug Run: 5 CPU tine for Debug run (sec): 8.79 

CPU tine for necessary compiles (sec) : 3.07 
a)1.5l b ) 1 . 5 6 c) a) e) f) g) 



Man hours for this Debua Run : .9 (excluding overhead) 

VAN 1) 



STEP! PROGRAM 
* ! PART 
1 
1 


OBJECTIVE AND EXPECTED 

RESULT 


ACTUAL 

RESULT 


DA y ! HOURS ! ERROR 
TIMES/STEP! ^ 

i i 

i < 


COMMENTS 
A NO CODED 
ERROR TYPES 


1 

1 












1) Record when 


i 

i 












error 


i 

i 












occurs 


1 ! U t i 1 i - 


Test calculations of num- 




2/05 


.5 






! ties 


ber of incut cards for 




1500 








• 

i 


maximum string length 












i 

i 


( i . e • 25b characters) 


O.K. 










t 

i 


(program should read 4 












i 

* 


cards) 












2 ! Palin- 


Test detection ana writ- 












! drome 


ing of a oal idrome of 


run 


1500 




a 1 


C 2 8 


! Check 


maxi mum 1 engt h 


error 










i 

* 


(PALI DROME COUNTER should 












■ 

i 


be 1# pal idrome should be 












i 

• 


detected and printed as 












* 

• 


d e s i g n e d ) 












t 

i 






2/05 








i 

i 

i 






1530 








i 

3 ! 


Repeat steps 1 and 2 


O.K . 


2/05 


.4 






4 ! Pal in- 


Test detection and writ- 




1530 








| drome 


ing of maximum number of 












! check 


palindromes of minimum 












i 

i 


1 engt h 


O.K. 










i 

i 


(PALINDROME COUNTER 












i 

• 


should be 125, all 120 












i 

i 


palindromes should be 












i 

* 


listed as designed) 












5 | Pa ) i n- 


Test for string without 












! drome 


pal indromes 


TEXT3 


1530 




42 


83 


! check 


(TEXT3 should be printed) 


not prin- 


1530 




43 


C28 


i 

i 




t ed 










i 

i 






2/05 








i 

i 






1600 









Remarks: "run error” indicates termination due to excess of array limits. 
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annex d 



Page a 0 f a 



WORKSHEET FOR DEBUGGING PHASE 
PROJECT 3 : 1 DEBUG Run 3 : 4 

Begin of Debug Run (day/time) : 2/05/1600 

End of Debug Run (oay/time) : 2/05/1700 

u of Debug Steps incl. in Debug Run: 2 CPU tine for Debug run (sec): *22 

CPU time for necessary comoiles (sec) : 1.67 



a ) l .67 b ) c ) c ) 


e) 




f ) 


g) 


Man hours for this Oeoug Run : .5 (excluding overhead) 
















VAN 


l ) 


STEPJPR0GRAM 


OBJECTIVE AMD EXPECTED 


ACTUAL 


DAY 


HOURS : ERROR 


COMMENTS 


u ; PART 


RESULT 


RESULT 


TIME 


/ster : 


1 M 


AND CODED 


% 

i 








i 


) 


ERROR TYPES 


i 

i 












1) Record wn« 


i 

% 












error 


• 

• 












occurs 


l ! I n i t i - 


Test for inDut of illegal 




2/05 


.5 






! a 1 i za- 


String length 


O.K. 


1600 








! t i on 


(program should print a 












» 

i 

« 


-arning and terminate) 












2 J Pal in- 


Test for overlaooina Da- 












| drome 


1 indromes 












1 check 


(should be recorded as 


O.K. 










» 

• 


two oa l i nd rome s ) 




2/05 








• 

i 






1630 
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ANNEX El 



Page 1 of 2 



ERROR LISTING 



PROJECT a : 1 



Begin of Project (day/time) 
End of Project (day/time) 
Man hours for total project 



1/28/1040 
2 / 06/1620 
21 .6 



ERROR 

a 


PHASE 

in which 

ERROR was 
dis- 
covered 


PHASE 
in which 
ERROR was 
made 


ERROR 

TYPE 

(see ANNEX F) 


TIME 

scent to 
solve the 
ERROR 
( ,M an 
min.) 


a of OTHEP 
STATEMENTS OP 
PAPTS OF THE 
PROGPAM 
AFFSCTEO 


1 


Design 


Design 


03 


5 


Data i nou t on 1 y 


2 


Design 


Design 


03 


5 


Data printout 


3 


Cod i ng 


Coding 


C 23 


5 


Indexing within 












subseauent sub- 












routine 


4 


Coding 


Coding 


C23 


3 




s 


Coding 


Coding 


CIO 


2 




6 


Coding 


Cod i nq 


C 8 


5 




7 


Cod i ng 


Ceding 


C 6 


5 




8 


Coding 


Codi ng 


C23 


5 




9 


Coding 


Coding 


C2 3 


10 


3 statements 


10 


Coding 


Design 


03 


30 


Whole subrou- 












tine affected 


1 1 


C o d i n g 


Codi n g 


C 1 


10 


Whole subrou- 












tine affected 


12 


Coding 


Coding 


C 1 1 


5 




13 


Coding 


Coding 


Cl 1 


3 




14 


Coding 


Coding 


Cl 7 


5 




15 


Coding 


Coding 


C 2 7 


5 




16 


Coding 


Coding 


C 1 5 


5 




17 


Coding 


Coding 


C 4 


5 




18 


Coding 


Coding 


C20 


5 




19 


Coding 


Coding 


C 2 3 


5 




20 


Cooing 


Codi ng 


C 12 


5 




21 


Coding 


Coding 


C 19 


5 




22 


Coding 


Codi ng 


C24 


10 




23 


Coding 


Coding 


C 1 2 


3 




24 


Coding 


Coding 


C 2 4 


15 


Whole subrou- 












tine affected 


25 


Oebugq i ng 


Codi ng 


A 1 


10 




26 


Debugqi ng 


Coding 


C 5 


5 




27 


Oebugg i ng 


Coding 


C 1 2 


5 
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ANNEX El 



p aae 2 of 2 



ERROR LISTING 



PROJECT u : 1 



Begin of Project (day/time) 
End of Project (day/time) 
Man hours for total project 



l/28/ioao 

2 / 06/1620 

21.8 



ERROR 

* 


PHASE 

in which 

ERROR was 
dis- 
covered 


PHASE 

in which 

ERROR was 
made 


ERROR 

TYPE 

(see ANNEX F) 


TIME 

spent to 
solve the 
E PROP 
(Man 
m i n . ) 


n of OTHER 

statements OR 

PARTS OF THE 
PROGRAM 

affected 


28 


Debugq i nq 


Coding 


A2 


5 




29 


Debuga i nq 


Coding 


C23 


3 




30 


Deougg i ng 


Cod l ng 


C 1 2 


5 




31 


Oebugg i nq 


Coding 


Cl 1 


3 




32 


Debugq i ng 


Coding 


A 1 


3 




33 


Oebugg i nq 


Coding 


C23 


5 




3a 


Debuqg i ng 


Cod i ng 


A2 


5 




35 


Oebugg i nq 


Design 


C28 


10 




36 


Deougg inq 


Des i qn 


Dll 


25 




37 


Debuqg i nq 


Coding 


C 8 


10 


Writ inq of o a - 
1 indromes 


38 


Deougg i ng 


Coding 


C27 


5 




39 


Debugq i nq 


Cod i ng 


C20 


10 


W r i t i n q of p a - 
1 i nd r ones 


ao 


Oebugg i ng 


Coding 


C23 


20 


Whole subrou- 
tine affected 


a l 


Oebugg i nq 


Coding 


C28 


15 




a 2 


Debugq i ng 


Oebugg i ng 


83 


3 




4 3 


Debugq i nq 


Cod i ng 


C28 


15 




ua 


Testing 


Des i qn 


D9 


20 





76 



ANNEX E 2 Page 1 of 2 

ERROR LISTING (COMMENTS) 



ERROR 

n 


DAY 

TIME 


comments 

(EVIDENCE, THOUGHTS, *HY WAS THE ERROR MADE? 
WHY AMD HOW WAS THE ERROR DISCOVERED? 

ERROR BLOCKING, etc.) 


l 


1/28 


Lack of concentration (step left out# which the programmer 




1505 


was aware of ) 


2 


2115 


This design step was postponed and forgotten later on. 


3 


2205 


Programmer did not cneck the manual* (Discovered while 


a 


2210 


examining the solution of a similar Drool em in an old Droqram.) 
same as 3 


5 


1/30 
1 3ao 




6 


1U00 




7 


iao5 




8 


l a 30 


error was found while examining the list of error types 


9 


1500 


in connection with errors 3 and a. 

Error was found *hile reading the programming manual. 


10 


1510 


Errors 10 and ll discovered during a desk test. 


i i 


1630 


see 10 


12 


1710 


(wrong indentation could lead to programming errors) 


13 


1710 


see 12 


U 


1720 




15 


1725 




io 


1/31 

1030 




17 


1050 


This statement was inserted between the wrong lines. 


18 


1 t 10 


It was detected in this early stage of the project 
because the programmer was ounching cards himself, which 
allowed him to review his source statements. 

Programmer did not look at the declaration of the variable. 


19 


1115 




20 


1120 


(missing mandatory declaration) 


21 


1125 


(wrong format) 


22 


1 130 




23 


1135 


same as 20 


2a 


l 1 ao 




25 


1/31 

1335 


special character ( n ) was expected to be necessary due to 


26 


1335 


similarity with other uses of hRITE/toRITEON, however this 
assumption was wrong 

(previously declared variable declared twice) 


27 


1335 


same as 20 


28 


1335 




29 


1335 


Programmer did not check the programming manual. 


30 


1335 


see 20 


3 1 


1335 




32 


1335 
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ANNEX E2 



Page 2 



of 2 



ERROR 

u 



33 

3a 

35 



36 

37 

38 

39 

ao 

a l 
a£ 
a3 

aa 



ERROR LISTING (COMMENTS) 



DAY ! COMMENTS 

TIME! (EVIDENCE, THOUGHTS, AH Y WAS THE ERROR MADE? 
! WHY AND HOW WAS THE ERROR DISCOVERED? 

' ERROR BLOCKING, etc,) 



1/31 
1806 
1806 
2/0a 
1 122 

2/05 
I3a5 
l a20 
l a20 
l a20 
2/05 
laas 
2/05 
1500 
1530 
1530 
2/Ob 
1525 



Error could have been avoided by looking at previous block of code. 

Porgrammer had correct design conceot but did not coae 
cor rec t 1 y . 

Boundary conditions had not been checked before. 



Trivial error .as n a d e , cecause orogrammer was tired. 

Design was not stated well, 
sane as 90 
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ANNEX F 



FINAL STATISTICS 



PROJECT a 1 



FINAL STATISTICS 
Project name 5 PALINDROMES 
Short description: 

Given an inout string of length n, -here 2 <= n <= 256, 
fina all occurences of oal indromes and list t n e m wi tn 
resoect to their beginning and ending ocsi t ions on the inout 
card. 

Inout: via o u n c h cards 
Qutout: via line printer 

Quant i tat ive measures: 

1. a of source statements : 14! 

2. Total man hours for project : 21.6 

3. Man nours scent in 



a ) Design : 5.0 

C ) Coding : 7.0 

c) Debugging : 4.0 

d) Testing : 5.8 

4. CPU time for comoiles: 20.22 sec. 

5. CPU time for debug runs: Q . 1 1 sec. 
o. CPU time for test runs: 13.98 sec. 

7. a of test and deoug runs: 5 

8. a of test and debug steos: 23 

9. a of errors found? 4 cj 

10. Total man hours used to correct errors: 6.73 
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ANNEX F 



final statistics 



11. Error Detection: 

a) Mean time between error detections: 20.3 man min. 



a of 
Errors 
r 



2a 



****** 



20 t 



10 * 



5 * 



* 

* 

* 

* 

******* 

* * 

* * 

* * 

* ******* 

* * * 

* * * 

* * ******* 

* * * ******* ************* ****** 



+ + - — - - + ... > + — > 



0 -15 



(measured 



16-30 31-a5 06-60 61-75 76-9Q 91-105 106-130 151-175 man m 

o f 

project 

TIME BETWEEN EPROP DETECTIONS 
from coint in time of detection of previous error) 



n . 

i me 
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ANNEX F 



FINAL STATISTICS 



12. Error Correction: 

a) Mean time to correct an error: 7.8 man min. 



# of 
Er ro r s 
c o r r ec t ed 
T 



20 + 



15 + 



10 + 



5 + 



■+-+-+-+- 

2 3 a 5 



1 0 



15 



20 



TIME TO CCRPECT ERRORS 
(measured from point in time of detection) 



25 30 

man minutes 
of project time 
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ANNEX F 



final statistics 



13. 


When 


errors 


were 


founa: 












a ) 


a 


of 


e r ro r s 


found 


during 


design phase: 


1 


a 


2.3 


X 


b) 


9 


0 f 


e r ro r s 


found 


during 


design review! 


1 


= 


2.3 


X 


c ) 


a 


0 f 


e r r o r s 


found 


during 


coding: 


22 




50.0 


X 


d) 


X 


0 f 


e r ro r s 


found 


during 


deougq i ng : 


19 




43.2 


X 


e ) 


9 


0 f 


er rors 


found 


during 


writing of 




















test orocedu res: 


0 


= 


0.0 


X 


f ) 


9 


0 f 


errors 


found 


du r i ng 


testing: 


1 


= 


2.3 


to 



ay 



14. 


When 


er ror s 


were 


maoe : 




a ) 


a 


0 f 


errors 


made 


during 


design phase: 


0) 


9 


o f 


errors 


made 


du r i ng 


design review: 


c) 


9 


o f 


er ror s 


maoe 


Ouri nq 


coding: 


d ) 


9 


o f 


e r r o r s 


made 


du r i nq 


debuqo i ng : 


e) 


9 


o f 


e r ro r s 


made 


during 


writing of 












test procedures: 


f ) 


9 


o f 


e r r o r s 


made 


du r i na 


testing: 



5 = 11.45 
0 = 0.0 % 
38 = 8 6 . a x 
1 = 2.3 ?. 

0 = 0.0 
0 = 0.0 



aa 
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annex f 



FINAL STATISTICS 



1 5 • TIME history graphs : 



» of 
Errors 
T 

i 

i 

! Error Tyoes: 

I 4; Clerical error 

{ 6: Oeougginq error 

J C: Coding error 

! 0: Oesian error 

6 + A 

! C 

! C 

5 f C A 

! C C 

! C C C CC C 

! C C C CCC CA C 9 

1 ♦ 0 OC CCCCCCC CCC ACOCCCC 0 

123^5 10 15 20 man hours 

. .Design. • .Coding Debuoginq. .Test i ng . . 

(Software Development Phases) 

NUMBER OF ERRORS FOUND VS PROJECT TIME 



U of 
E r r o r s 

r 

* 

* 

* * * * 

* * * * *********** 

* ***** **** ************ * 

12345 10 15 20 nan hours 

NUMBER OF ERRORS CORRECTED VS PROJECT Time 



5 ♦ 
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ANNEX F 



ERROR CATEGORIES AND TYPES 



1 « Design Errors 

The foilwing tyoes of errors apply to both categories 
"System Design Errors" and "Program Design Errors": 

D1 : Communication Error 

D2 : Design Negligence 

D3 : Forgotten Cases or Steos 

D4 : Timing Problems 

D5 : Errors in I/O Concepts 

06 : Data Design Error 

D7 : Initialization Error 

08 : Inaaeauate Checking 

DR : Extreme Conditions Neglected 

010: Seouenci nq Error 

Dll: T nde x inc Error 

D12: Lood Control Errors 

013: M isuse of Boolean Expression 

D14: Mathematical Error 

D 1 S : Reoresen t at i on Error 

016: M i sunoe r s t and i ng of Problem Spec i f i ca t i on s 

D 1 7 : Other Desian Errors 

2, Coding Errors 

Cl : Mi sunder st anhi ra of Design 

C 2 : Negligence 

C3 : 1/0 Format Error 

C4 : Misplaced Data Declaration 

C5 : M ultio1e Data Declarations 

C6 : Missing Data Declaration 

C7 : Inadeauate Data 

C8 : Initialization Error 

C9 : Error in Parameter Passing 

CIO: Inadeauate or Forgotten Checking 

C 1 1 : Level P r ob 1 ems 

C 1 2 : M issina Declarations of Block Limits 

C 1 3 : Case selection error 

C 1 4 : GO TO Problems 

CIS: Comment Error 

C16: Forgotten Delimiter 

Cl7: Inconsistency in Naming 

C18: Wrong Use of Nested IF Statements 

C 1 9 : Index i ng Error 

C 2 0 : Inconsistent Use of Variables or Data 

C21: Seauencing Error 

C22: Flag Usage Problems 

C23 : Syntax Error 

C24; Loop Control Error 

C25 : Incorrect Exit from Subroutines 

C26: Language Usaae Problems 
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ANNEX F 



ERROR CATEGORIES AND TYPES 



C 2 7 : Forgotten Statements 

C23: Representation Error 

C2R: Control Seauence Error 

C30: Incorrect Subroutine Usage 

C31: Other Coding Errors 



3. Clerical Errors 



A 1 : Manua 1 Error 

A2 : Mental Error 

A3 : Procedural Errors 

A 4 : Other Clerical Errors 



4 . Deouqging Errors 

31 : Inacoroor i at e Use of Oeouqginq Tools 

32 : Insufficient or Inacoroor i ate Selection 

of Test Cases or Test Data 

33 : Mi s i nterorot at i on of Debugging Results 

04 : Misinterpretation of Error Source 

05 : Neq 1 i gene e 

Qo : Other Oebuqging Errors 

5. Testing Errors 

Ti : Inadequate Test Casefs) or Test Data 

T2 : m i s i n t e ro r et a t i on of Test Results 

T3 : Misinterpretation of Program Specification 

T 4 : Negligence 

T5 : Other Testing Errors 
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ANNEX G 



P ^ge lof9 



OIRECTED GRAP^ REPRESENT A P ION 



PROJECT a : i 

Program oart : INITIALIZATION 



COMPLEXITY MEASURES: 



number of statements: U 

NUMBER CF NCCES: 5 

NUMBER CF ARCS : __ 5 

NUMBER C p PATHS: 2 

CYCLCMATIC NUMBER: V ( G ) = 2 



REACHABILITY CF NCOES: 



NO CE 1 

NOCE 2 

NCCE 2 

NOCE A 

NCCE 5 



SUM: 7 , C C C C QC 

REACHABILITY INCSX 
OF OIRECTEC GRAPH : 



1 .ROOOOJ 
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ANNEX G 



Race 2 of R 



DIRECTED GRAPH REPRESENTATION 



PROJECT 2 : t 

Program oa^t « BLANK LINES 



COMPLEXITY MEASURES: 



NUMBER OF STATEMENTS: o 

NUVRER Cr NCC5S: 6 

NUMBER CF ARCS : 6 

NUMBER CF PAThS: 3 

CYCLCMATIC NUMBER : V{G)= 2 



REACHABILITY CF NCCES: 
i 

1 
1 

2 
1 
2 



c.CCCOCC 
REACFA5 IL ITY INDEX 
C c 0IRECT5C GRAPH: 1.232333 



NGCE 
NCCS 
NGCE 
NC Cc 
NCCE 
NC CE 



SUM : 
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ANNEX G 



P aoe 3 o f 9 



DIRECTED GRAPH REPRESENTATION 



PROJECT a : t 

Program oart : T E X T 1 , TEXT 2 , TEXT3 



COMPLEX I T Y MEASURES : 



NUMBER OF STATEMENTS: a - 1 l 

NUNS ER CF NCCSS: 2 

■NU V S £ R CF ARCS : I 

MJ>6£R CF PATHS: 1 

CYCllVATIC tNUVSEF: V(G) = I 



REACHABILITY CF NODES: 
NQCc i : 1 

NuCH 2 : I 



SUV: 2.CCCCCC 

REACHAP T L i T Y INCEX 
CP DIRECTED GRAPH: l.COCOCC 
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ANhtX G 



P 30® 4 o t 0 



0 I p EC T ED GRAPH R£PPE3ENTi T ION 

PROJECT s : 1 

Prooram oaf f : READ AND •"< B t T E INPUT CAROS 



COMPLEX IT r MEASURES: 



MUMbE-i CP STATEMENTS: IS 

XUT'EER CP ACCSS: o 

iNUf'BcP CP ARCS : 7 

NU'-BER CF F J T t- S : b 

CYCLCi v ATIC NUi v BcF : V(G)= 3 



R E AC I-A2 I L ITY CF .NCCES: 



NCCE l : 


l 


NCCE 2 : 


c 


NCCE 2 : 


c 


NCCE 4 : 


2 


NCCE 5 : 


c 


NCCE 6 : 


6 


SUV : 


2 6 .CCCCC 


REXCrASiL ITY 


INCH* 


CF CIRECTSC 


C-PAPH : 




8Q 



ANNC< U 



i^aoe 5 of 9 



DIRECTED GkAPh REPRESENT A r ION 

PROJECT a : 1 

Program oart : >n RITE ALL p a l I <\ 0 R 0 f ,J E S 
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ANNE* G 



P age oof Q 



0I c ECT£0 GRAPH REPRESENTATION 



PROJECT r : l 

Program oart : PALINDROME CHECK 



a) 2 of nodes: 10 

b) 2 of arcs : 13 

c) 2 or statenenrs: 7 

c) 2 of oatis: * 

e) 3 eecnaoi 1 i tv: ♦ 

f) Cvdorat i c nu^cer: 5 
* Numoer of cat^s 

ana reacnaoi 1 i f y 
are very 1 arc? . 
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ANNEX G 



Page 7 of R 



DIRECTED GRAPH r -?EPR£ SENT A T I CM 



PROJECT 3 : 1 

Program oart : CONTINUE CHECKING 



COMPLEXITY MEASURES: 



NUMBER CF STATEMENTS: 15 



NUMBER CF NCCSS: 3 

NUMBER CF ARCS : 9 

NUMBER CF PATHS: 7 

CYCLCMAT IC NUN BER : V ( G ) = 



REACHABILITY CF NCOSS: 



NCCE 
NCCE 
NO CE 
,\C CE 
NCCE 
NCCE 
nClc 
NCCE 



SUM: AC.CCGQO 

REAChAB IL I T Y INCEX 
CF CIR6C7EC GRAPH: 
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ANNEX G 



ace 



P o f R 



p 



DIRECTED GRApw REPRESENTATION 



PROJECT 5 : l 

Proaratr o a r t : RECORD PALlNQRQME 



a) 5 of no^es : ° 

b) 3 of arcs : 12 

c) 3 of statements: 2 l 

d ) - of oacns: * 

e) C eacnacn I i ty : * 

f) Cyclomatic nurcer : 5 
* Nu-noer of oat^s 

ana r e ac lao i 1 i tv 
are very I ane. 
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ANNEX G 



J aoe * of 0 



OIPECTEO GWAPm PE pc, F5ENT A f [ON 



PROJECT 3 : l 

Program oart : M A I N 



COMPLEXITY MEASURES: 



NUMBER QF STATEMENTS: 0 

MJ N*3 ER CF NCC6S: 6 

NU^ccR C f ARCS : 6 

MJf'EER Cr PAThS: 2 

CYCLCYATIC NUPEE-: V(G) = 



REACHABILITY Cr NCCES: 
1 
1 
1 
1 
2 



a.ccoaGe 

Pc AC I- A e IL ITY INC 5 X 
CF CIRECTEC GRAPH: 1 



NOCE 

,NCCE 

NCCc 

NCCE 

NGCE 

NOCE 



jL P : 




a 4 



- 9'4 - 



ANNEX H 



Page 1 



of 2 



TEST PHASE DESCRIPTION 



Project * : 1 

Test run u : 1 Including 7 Test Steos 

Begin of Test Cdav time) : 2/06/1430 End of Test (day/time) : 2/06/1620 

CPU time for necessary compiles (in sec.): 5.18 

a) 1.69 b) 1.71 c) 1.78 d) e) f) g) 

CPU time for TEST run (sec) J 13.98 

Han Hours in Testina : 5.8 (including oreoaration of tests) 



1 ) 



TEST 

STEP 


OBJECTVE 


EXPECTED 

RESULT 

(TOLERANCE) 


ACTUAL 

RESULT 


ERROR 


DAY 

TIME 


COMMENTS 

AND 

CODED ERROR 

types 














1 ) Record when 














error occurs. 


1 


Check program for 


All palindromes 






2/06 






s m a 1 1 pal indromes 


are oet ec t ed and 






1430 






( a ) pa 1 i nd r ome o f 


recorded as spe- 












1 ena t n 2 


cified bv pro* 












(b) palindrome of 


gram soecifi- 












1 eng t h 3 


c a t ions 


O.K. 








2 


Check for la rge 














pal i dromes 


same as 1 












(a) palindrome of 




progr . 


44 


1525 


09 




1 eng t h 255 




error 










(b ) pa 1 i ndrome o f 














length 25o 












3 


Test cor rec t i on 














Repeat s tep s 1/2 


same as 1 


j ob 




1550 


Implementation 








t e r m i - 






error (time 








na t ed 






estimate exceeded) 


4 


Test correction 














Repeat steps 1/2 


same as 1 


O.K. 








5 


Check palindromes 


should be de • 












which cross carp 


dec t ed ana prin- 












boundaries 


ted as spec i f i ed 


O.K. 








6 


Check for palin- 


same as 5 












dromes of various 














lengths and sizes 




O.K. 








7 


Check for invalid 


flaming and ter- 








See notes 




input (string 


mination 


O.K, 




2/06 






1 engt h = 1 ) 








1 620 





Notes: Other invalid inouts were exhaustively tested durinq 

debugging phase. 
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ANNEX h 



Page 2 of 2 



Choice of Test Oat a : 

Some test data were chosen to check the program for 
conditions. Other test data were selected to check 
size and numbers of palindromes. 



bounda r y 
random 
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APPENDIX 8 



PPOJECT DESCRIPTION 

Project ft : 2 

Project title: PATH ANALYSIS IN DIRECTED GRAPHS 

Program sier : HOF FM. ANN 
Programming Language : ALGOL 

Programming environment: I8M/360/67,OS/MVTfBATCH 

and TIME SHARING (CP/CMS) 

Design notes : see ANNEX A 

Program listing : see ANNEX B of APPENDIX C 

Coding notes : see ANNEX C 

Debuqginq notes : see ANNEX D 

Error Listing : see ANNEX E 

Final statistics : see ANNEX F 

Graphical reoresentation : see ANNEX G 

Test phase description: see ANNEX H 

Starting date: i MAR 77 Ending date: 17 

EXPERIMENT DESCRIPTION 
1. Project description: 

A. General Description 

This program is designed to find and write all 



MAR 7 7 



ooss i b 1 e 



oaths of minimal length (i.e. if a directed graph contains 
loops? no loop may be traversed more than once in succes- 
sion) through a directed graph. 



B. Inout Descriot ion: 



(a) Batch 
Card 1 : 1 



Card 2 : t 



Card 3 : 



Card 4 : 



Process i ng : 

in card column 1 followed by any text 
(this text will identify the graph and will 
be printed as a headline preceding the input 
documentat i on) 

in card column 1 followed by one blank; 
where t specifies the type of Path listing 
de s i red . 

t=0 -> all oaths are listed during analysis 
(recommended only for a very large 
number of oaths; i.e. more than 500 
paths expected) 

t is nonzero -> paths are listed after 
analysis has been completed 
(any additional information will 
not be interpreted by the program) 
in care column 1 followed by one blank 
where s specifies the number of digits 
of the largest node name used in the directed 
graph and 1< = s < = 3 

(any aoditional information will not be 
interpreted by the Program) 
n si s2 ... sn 

where 1 is the node name of the entry node 
and may be punched in any column 
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n is the number of successors 



and 0 <= n <= 1 4 

si ( i = 1* 2 f • • • n) specifies the name 
of the ith sucessor of node 1 
Card 5 thru Cara m+3: 

where m specifies the number of nodes in 
in the directed graph 
Format! j n si s<? sn 

where j specifies the node name of the node 
being described on this card 
and 2 < = j < = 999; j may be punched 
in any col umn . 
n is number of successors 
and 0 <= n < - 1*4 

si ( i = 1 t 2 . . . n) specifies the name of 
the iths successor (1 <= si <= 999) 

if n=0 -> no entry necessary 

NOTE: 

(a) All entries are integers and must be separated by at 
least one blank, 

(b) All information about any node must be on one card, 

(c) Card column 80 must contain a blank character or an 
additional blank card has to be inserted. 

Card 4 + m : 99999 followed by at least one blank 

(this input indicates that all information of 
the oreceeding directed graph has been input) 
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NOTE: Following this description any number of directed 
graphs may be specified and submitted to the program. After 
the last descriotion cf a directed greDh a termination card 
has to be added which contains any integer different from 
"1" in card column 1. 



Sample Input : 



1 

0 

2 

1 

2 

3 

a 

5 

99999 

1 

1 

1 

1 

2 

3 

a 

5 

99999 

-1 



(FIRST GRAFF) 
TOGGLE 

WAX MODE NAME 

1 2 

2 3 9 

0 

1 5 

0 

(SECOND GRAPH) 
TOGGLE 

WAX NODE MARE 

3 2 

a 

1 5 



TERMINATE 



(b) Input under CP/CMS: 

The input seauence under CP/CMS is similar as under 
batch orocessina. There is only one exception: Any other 

input than the i nout requested by the program will terminate 
the program. The user can only input integers, additional 
blanks are not allowed. 

Under CP/CMS instructions are displayed at the terminal 
which makes the use of the program almost sel f *exol anatory . 
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NOTE 



(a) Login with 540k 

(b) Follow the directions of the program orecisely. 

(c) Input only inteqers. 

C . Error Messages : 

All error messages are sel f-exol anatory , 

Errors 3, 8 , 9, 10 refer to invalid or incomplete input. 

Errors 8 and 9 will cause the program to terminate. 

Error 8 indicates the limit of the program. If error 2 oc- 
curs it is most likely that the directed graph has an infin- 
ite number of paths. The user may try the same input using 
H 0" as input on the second card. Error 1 will only occur if 
more than 30 nodes nave been specified having 14 successors 
each. 

All other errors will indicate an error which has not 
been found during testing and debugging or will be due to 
some abnormal usage of the program, In either case it might 
be possible to locate the Problem by examining the program 
listing or retrying the program. 
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2. Programmer's background: 

a) Experience in programming: 



Oct 


1970 - May 


1971 


Programming courses 


May 


1971 - April 


1972 


Module Programmer 


May 


1972 - June 


197a 


Work in Test and Simulation Depart 



men t at t He 

NAV AL COMMAND AND CONTROL SYSTEMS 
COMMAND (FEDERAL GERMAN N/fVY) 

Testing of tactical real time systems 
March 1975 - Jan 1977 Student at the NAVAL POSTGRADUATE 

SCHOOL, Monterey, Computer Science 

b) Experience in testing: 

Two years of work in testing ana simulation. 

c) Experience in the area of the given problem: None. 

d) Experience in the proqamminq language being used: 
Experience over a period of 18 months in more than 10 

programming projects. (Total number of source statements 
produced during that time was more 4000.) 

3. Psychol ogi ca 1 factors: 

a) Dio the programmer like the project? Yes. 



b) how does the orogrammer like the programmina 



1 anguage? 

Favorite programming language. 



c) Was the programmer satisfied by the way the problem 
was soec i f i ed? 

Only minor criticism. 



d) How did the programmer like the orogramming environ* 
men t ? 

The facilities (study room, card punch room) were not 
conducive to efficient orogramming because of restricted 
space, bad lighting and noise. 

e) Other factors: 

The recording of the experiment’s data during the pro- 
ject affected soeed and concentration considerably. 
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4, Comments on Documentation 



For the documentation of each software development phase a 
special documentation form has been devel ooed. These forms 
are designed to provide a firm quideline for the experiment 
programmer to record all data of interest for subseauent 
error analysis* 

- Begin and end of each step was recorded with respect to 
day and time. 

- Each error was recorded when it is discovered. The error 
was then identified by a uniaue error number (W2/.*.). 
Furthermore the time of discovery and the error type (using 
error tyces listed in ANNEX F) were recorded. 

- If appropriate/ comments about error discovery/ reason why 
the error was made/ etc. were documented in ANNEX E 2 * 

- For each error the phase in which the error was made/ the 
phase in which the error was discovered and the time spent 
to correct the error was recorded in ANNEX El. 

- For each step in any one of the software development 
phases the day/time of begin and end was recorded. In addi- 
tion# the time (in man hours) for each step was recorded. 
This excludes the overhead used for documentation of the 



exoer i ment data 



ANNEX A 



Page 1 of 5 



WORKSHEET FOR DESIGN PHASE AND DESIGN REVIEW PHASE OF PROJECT * 2 



STEP 

u 



PR08LEM AND PLANNED SOLUTION 

i 



ALTERNATE 

SOLUTIONS 



1 {Analyzing structures with 

! respect to minimum number of 
Joaths. 

{The minimum number of oaths 
{includes all oossible oaths 
{starting at the entry node (1) 
{and ending at any node which 
{has no successor. The minimum 
{number excludes all oaths ha- 
{ving reoetative traversals of 
{loops, such that any seauence 
!of arcs is traversed more than 
{ once i n a row . 



{ MAN 

DAY {HOURS 
TIME { /STEP 



ERROR 

* 



comment 



3/01 

1900 



12.0 



3/03 

2400 



2 



Oesign of Data Structures: 



! 3/04 { 


I De t a i Is 


: 1 1 o o j 


{see 


! 3/04 ; 


{Remarks 


! 1900 { 


• 

i 



Rema rk s : 

Two freelists are used to sucol y storage space to keeo information about 



oaths 


and 


the structure of the unaerl 


y i na 


di rec t ed gr aoh 








FREELIST 1 


(Ml) 








[451 


Items of R bytes 


each 


(32 Bits)) 


Items 












used 


for! 














< 2 


0 vt es > < 

• 


2 


Bytes > 



a) 



Successor 
Informat i on 
(el ement 


successor 


name 


{ Pt r 


next 


successor { 


i n forwa rd 
1 i n ked list) 






1 •••* 






b) 

f4 A A A a ^ 






• 




* 

i 


n cau cr O' 

successor 

lief 


reachabi 1 


i t y parent 


• 

ot r 


i 

successor list I 


1 1 3 L 










i 


c ) 

Pr ee 1 i s t 
Header 
(Item 0 ) 






• 




i 

i 




0 


! ot r 


next 


i 

free item { 

i 
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FREELIST 2 (’-'2 ) 



Items 
used for: 


120001 Items of 8 Bytes each (64 Bits)) 

• 

c 2 Bytes > < 2 Bytes 5 


a ) 

Node 

Informat ion 




» 

1 


pointer to original 


node name 


( duo 1 i c a t e ) 


otr to successor 


otr to o r edec e sso r 


D ) 

3 » k a job * 




» 

l 


ratnneaaer 
Inf ormat i on 


ptr to previous path! 


loath i oen t i f i c a t i on 

i _ 




otr to bea in of oat h J 


i 

! p t r to next oath 


c) ! 

Node 


i 

\ 

1 


1 

» 

1 


noo e i 

Information ! 
(original ) ! 

i 

i 

i 


[pointer to original 


node name ! 


3 of successors 

1 


i 

ot r successor list ! 

i 


i 

d) 

F r e e 1 i s t 






Header 


0 


0 ! 


i i c e“ w ; 


0 


otr next free item ! 
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Typical usage: 



a) Original node with successor list 






n ! pt r« 



r ! pt r > 

i •• i 



J ! p t r 

_ i _____ i 
i i 



> 



k! NIL 



rthere: 

i , j / k are node names 
r represents reachability of node i 
□ is the storage location within v 2 
NIL indicates the end of a linked list 

b) Ooubly linked list of path headers: 



<----o t r 

i 


! i i 


! <--* 

! 




i 


• 

i 

• 


itl { 


! Dtr 


i i 

o t r - - 






J ot r 


i 

• 

i 


i 

Dtr-- 




V V 



beginning beginning 

of oath i of oath itl 

^here i and i > 1 are path identifications, 

c) Nodes (duplicates) being elements of a oath 





• _ _ _ _ _ i i 

i • i 


T oredecessor of node 

i 

t 


original < — — — * 
o f i 


i i • 

i i i 




i ot t ( pt r--- 

■ _ „ i i i 

i i i i 

! T 

i i 






• 

• 

V 

1 

1 1 * I 


i 

i 

i 

i 

i 

• 


original 
of j 


1 • | 

1 1 I 

! pt r | pt r — - 


T 

i 

i 

i 



V 



.•ihere i and i are successive nodes on a oath. 
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WORKSHEET FOR DESIGN PHASE AMD DESIGN REVIEW PHASE OF PROJECT n 2 



STEP 

U 


problem AND PLANNED solution 


ALTERNATE 

SOLUTIONS 


! man 

DAY {HOURS 
TIME I /STEP 


ERROR 

u 


C 0 M MEN T 


3 


TOP - DOWN Design: 

Define the following orogram 




3/05 

1500 









Da r t s : 

-Data st rue tures 
-Primitives to suooort data 



struct u res 

-Utilities to suooort data 
s t rue tures 

-Logical orocedures to suooort 
analysis of directed qraohs 
-Subroutines to suooort analy- 
sis of oossible oaths through 
a directed qraoh 
- I nou t of data 
-Ou tout of data 
-Error messages and relevant 
diagnostics to suooort error 
error investigation 



3/05 

loOO 
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WORKSHEET for DESIGN phase and design REVIEW PHASE OF PROJECT u 2 











! man , 


i 

i 




STEP 


PROBLEM AND PLANNED SOLUTION 


alternate 


DAY 


} HOURS ! E^ROR 


COMMENT 


n 




SOLUTIONS 


TIME J /STEP S » 




a 


Define Data Structures 




3/05 










a) Use Freelist (Ml) of 451 

4 byte woras used to orovide 
information about successors 
of parent nodes (each of max, 
30 parent nodes may nave up to 
1 4 successors ) 

b) Use Freelist (M2) of 20001 
q byte items to Weeo informa- 
about original nodes* oath 
heaaers and oossible oaths 




1600 


.5 








through the directed graph 
being analyzed. 

(each item is designed to hold 




3/05 










4 integer values oetween 0 ana 




1630 






see also 




65535) 










r ema r < s 
on page 1 


s 


Design of primitives ana 




3/05 










utilities to sucoort data 
s t rue t ures 

a) Primitives for Freelist 1: 
-Initialize Freelist 1 
-Carl/ Cdrl (retrieval of 




1630 


5.5 








i n t eger values) 














-Setcarl/ Setcdrl (set values) 
-Allocatel# Freel (allocate/ 
free el emen t s of Ml) 




3/05 
20 30 










b) Primitives for Freelist 2: 




3/ Oo 










-Initialize Freelist 2 
-Car2/ Cdr2 (retrieval of 




1330 










\ n t ege r va 1 ues ) 














-Setcar2/ Setcdr2 (set values) 
-Allocate2/ Free2 (allocate/ 




3/06 










free elements of M2) 




1500 
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WORKSHEET FOR DESIGN PHASE AND DESIGN REVIEW PHASE OF PROJECT # 2 



STEP 

U 


PROBLEM and planned solution 


ALTERNATE 

SOLUTIONS 


DAY 

time 


MAN : 

HOURS : ERROR 

/step: * 


6 


Define Algorithms for Utili- 




3/Oc 


1 

1 

1 

1 




ties to suooort structures 




1300 


1 

1 




using Ml: 






.5 1 




•Name; Setname 






1 

1 




-0rother# Aaoorother 




3/0b 


% 

1 




-Son(i) (get name of ith son) 




1330 


1 

1 

• 


7 


Define Algorithms for Utili* 




3/07 


1 

I 

1 




ties to suooort structures 




1000 


1 

1 




usina M2 : 






2.0 : 




a)Suooort of linked list of 






1 

1 




oath headers: 






« 

• 




-Set Path ID# -Path ID 






• 

1 




-Linkpath(forward/fcackward) 






1 

1 




-Initialize Pathlist 






1 

1 




-Next oath# -Previous Path 






1 

1 




b) Supoort of oath structures: 






1 

1 




-Name of (retrieve node name) 






1 

I 




-Predecessor 






1 

1 




-Successor 






« 

1 




-Numoer of Successors 






1 

1 




-Linking of nodes( forward and 






1 

1 




backward ) 






1 

1 




-Duo 1 i c a t e ( s e t duplicate node) 






1 

1 




- Addnode 






1 

1 




-Quo licate a Path 






1 

1 




- Imo 1 emen t a t i on of alternative 




3/07 


1 

1 




oath 




1300 


1 

1 

1 


3 


Define Procedural Algorithms: 




3/03 


1 

1 

1 




-Remove a Path 




1500 


.5 ; 




-List Path 






1 

• 




-List all Paths 






• 

1 




-Find End of a Path 




1505 


: 17 




-Find Original 




3/08 


1 

1 








1530 


• 

1 

1 

1 

1 

1 

1 

1 



COMMENT 



DP former 

analysis 

did not 

include 

trivial 

case 



no 
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WORKSHEET FOR DESIGN PHASE AND DESIGN REVIEW PHASE OF PROJECT a 











MAN 






STEP 


PROBLEM AND PLANNED SOLUTION 


alternate 


DAY 


HOURS 


ERROR 


COMMENT 


u 




solutions 


TIME 


/STEP 


a 




9 


Design Review: 




3/09 










Define global cointers: 

-Pa t hhead 

-Cur r en t ca t h 

-Lastpa t h 

-Cur rent node 

•Last occurenc e 

Define algorithm for oath 

analysis: 

-Set Paths 

Soecify logical algorithms: 
-Occurs Twice (node with same 




i ioo 


3.0 








name on same oath) 

-One hay (loop without alter- 
native branches ) 

-Match (same sequence of nodes 
duplicated on current Path) 
-Checkback (check for invalid 




3/09 










al temat i ves) 




1400 
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WORKSHEET FOR COOING PHASE OF PROJECT * : 2 
Beginning of Coding (day/time) : 3/06/1500 
End of Coding (day/time) : 3/10/1730 
Man hours : 26 (including punching of cards) 



1 ) 



C0( 

BEGIN 

OAY/TIME 


) ING 

END 

DAY/TIME 


PROGRAM PART 


ERROR ! DAY 

a ! T I M E 
1 
1 


COMMENT 












1) Record when error is 












detected. 


03/06/1500 




-Data Definition 




3 / Oo 








-P r i m i t i v es 


1 


1530 


C28 






-Error Handling 


2 


l5ao 


C28 






-Initialization 


3 


1620 


C 23 




03/06/ lo30 










03/06/1630 




-Punching cards 














a 


1740 


C28 




03/06/1800 










03/06/1830 




-Utilities 










03/06/2030 










03/07/1200 




-Utilities 




3/07 










5 


1 305 


DP 




03/07/1«00 










03/08/1000 




-Punching cards 




3/08 








( U t i 1 i t i e s ) 


10 


1000 


C12 (missing BEGIN) 








1 1 


1005 


C28 (mispelling of 












procedure name) 








12 


1005 


C 2 3 








13 


1010 


C 1 6 








14 


1025 


Cl 








15 


1030 


C 1 7 








16 


1 030 


C 1 7 




03/08/1200 










03/08/1530 




-Coding of Pro- 












cedural suorou- 


18 


1630 


D Q (design did not 






t i nes 






consider removal of 




03/08/1800 








first and last oath) 


03/08/ 1 P00 




-Coding of pro- 












cedural subrou- 










03/08/2000 


tines 








03/09/2000 




-Punching cards 














IP 


2130 


Dll (faulty design 












of index calculation) 








20 


2150 


CIO (faulty condition 












"Not Equal" instead of 




03/08/2200 
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WORKSHEET FOR COOING PHASE OF PROJECT n : 2 
Beginning of Coding (dav/time) : 3/06/1500 

End of Coding (day/time) • 3/10/1730 
Man hours : 26 (including punching of cards) 



1 ) 



COOING 


program part 


ERROR 


0 A Y 


COMMENT 


BEGIN 


END 




a 


TIME 




DAY/TIME 


DAY/TIME 




















1) Record when error is 












detected. 


03/09/1200 




-Coding of I/O 




3/OR 








subrout i nes 










03/09/1300 










03/09/1300 


03/09/1500 


-Pune h i ng cards 


3a 


1440 


Cl 1 


03/09/1500 




-Coding of Logi- 












cal orocedures 


35 


1530 


OP 








36 


1615 


OP 




03/09/1730 










03/09/2000 


03/09/2100 


-Punch i ng cards 


40 


2030 


C20 


03/09/2100 




-Cod i ng of path 












analyzing a 1 go- 
r i t hm ( "Se t 


42 


2200 


OP 




03/09/2300 


oaths'*) 








03/10/1000 


03/10/1100 


-Punching cards 


44 


3/10 

1040 


015 


03/10/1 100 




-Coding of I/O 












suorout i nes 










03/10/1200 










03/10/1630 




-Punching cards 


46 


1700 


OP 



03/10/1730 
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WORKSHEET FOR DEBUGGING Phase 
PROJECT a : 2 DEBUG Run a : l 



Begin of Debug Run (day/time) 5 03/06/1800 

End of Debug Run (day/time) ! 03/07/1000 



a of Debug Steps incl. in Debug Run: <1 


CPU time for 


Debug 


run ( sec ) : 1.62 


CPU time for 


necessary compiles (sec) 


: a. 33 










a) 0.89 b) 1.76 c) 1.68 o) 


e) 




f ) 


q) 


Man hours for this Debug Run : 4,0 (including preparation of 


debug 


run ) 










MAN 


1 ) 


STEP ! PROGRAM 


OBJECTIVE AND EXPECTED 


ACTUAL 


DAY 


|hours;error 


COMMENTS 


a 1 PART 


RESULT 


RESULT 


times/step: * 


AND CODED 


i 










i 


ERROR TYPES 


i 

i 












1 ) Record wh< 


i 

• 












error 


i 

* 












occurs 


1 J P r i m i - 


Get error free compile 


3 compile 


3/06 








J t i ves 




e r ro r s 


iai i 


1 .3 


5 


A 1 


J and 










6 


C 23 


! oa r t of 










7 


A 1 


! ut i J i - 






3/06 








1 t i es 






1 0 30 








2 | 


Repeat step l 


O.K. 


3/07 


1.2 






i 

• 






1110 








3 | P r i m i - 


Check initialization* 






0.5 






! t i ves 


allocation and freeing of 












t 

i 


items in both freelists. 












• 

» 


Check parameter implemen- 












t 

i 


t a t i on . 












* 

i 


Check writing of blank 












i 

» 


1 i ne s . 












t 

* 


(pointers of freelists 


UDoe r 


1250 




8 


A 1 ("*1 M 


• 

t 


must be set aporoori atel y / 


half of 








left out 


1 

• 


allocated elements must be 


a 1 located 








while pun- 


1 

t 


filled with zeroes* 


e 1 emen t 








ching cards 


* 

t 


allocated items must be 


of M 2 not 










1 

• 


uni inked* 


filled 










t 

I 


after freeing an element 


with 










• 

• 


becomes part of the free- 


ze roes 










• 

« 


list aaa i n * 












1 

» 


parameters should be set 












t 

1 


as designed) 












a ; 


repeat steps 1 and 3 


O.K. 


3/07 


1 .0 







1000 
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WORKSHEET FOR DEBUGGING PHASE 
PROJECT * : 2 DEBUG Run n : 2 

Begin of Debug Run (day/time) 2 03/08/1000 

End of Debug Run (day/time) : 03/08/1700 

U of Debug Steps incl. in Debug Run: 2 CPU time for' Debug run (sec): .46 

CPU time for necessary compiles (sec) : 2.23 
a) 2.23 b) c) c) e) f) g) 



Man hours for this Debug Run : 2.5 (including orecaration of debug run) 

MAN 1) 



STEP 

tt 


PROGRAM 

PART 


OBJECTIVE AND EXPECTED ! ACTUAL 

RESULT ! RESULT 

1 
• 


DAY : HOURS : ERROR 

TIME ! /STEP ! » 

1 

1 


comments 

AND CODED 
ERROR TYPES 






1 

t 

1 


1 1 

1 1 

1 • 


1 ) ‘Record w n en 






1 

1 

1 


1 • 

• 1 

1 • 


occurs 


1 


Primi- 


Check maximum number of ! 


3/08! ! 






tives 


allocations in both free- ! 


1000! : 


2 man hours 




and 


lists. ! 


! 0.3 ! 


scent in 




Part of 


(all elements exceot free-! O.K. 


• i 

• t 


preoarat ion 




utili- 


list headers are alloca- ! 


i i 

• i 


of deouq 




ties 


ted) 

• 


• i 

• i 

• i 


run 


2 


Error 


• 

Check error messages for | 


i i 

i i 

• • 






h and- 


exhaustion of each free- ! 


3/oe: o.2 ! 






1 i ng 


list. I O.K. 

• 

t 


1700! ! 

1 1 

1 1 
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WORKSHEET FOR DEBUGGING PHASE 



PROJECT * : 2 DEBUG Run A : 3 

8egin of Debug Run (day/time) : 03/08/2200 

End of Debug Run (day/time) : 03/09/1300 

A of Debug Steps incl. in Debug Run: 7 CPU time. for Debug run (sec): ,6 
CPU time for necessary compiles (sec) : 18.12 
a) 2.67 b) 2.75 c) 2.75 a) 2.43 e) 3.49 f) a. 03 g) 



Man hours for this Cecuq Run : 4.5 (including preoaration of debug cyn) 

MAN 1) 



STEP ; PROGRAM 
* ; PART 


OBJECTIVE AND EXPECTED 

RESULT 


ACTUAL 

RESULT 


OAYJHCURS'ERROR 

time: /STEP! s 
1 
1 


comments 

AND CODED 
ERROR TYPES 


1 

< 














1) Record when 


1 

1 














error 


t 

1 














occurs 


1 ! Primi- 


Get error free comoile of 


75 com- 


3/08 








tives/ 


all subroutines and ori - 


pile d i a - 


2200 


1 .0 






! Error 


mi tives excluding I/O sub- 


gno sties 










! hand- 


routines ana logical sub- 


( see 


2200 




21 


C23 


! 1 i n g / 


rout i nes . 


remarks) 






22 


C 1 7 


J Proc e- 












23 


C 2 7 


1 dura 1 












24 


A 1 


{ sub rou- 
















tines 

< 
















< 

2 : 

i 


Repeat 


steo 1 




2245 


0.5 


25 


C 1 7 


i 

3 J 

i 


Repeat 


step 1 




2310 


0.5 


26 


A 2 


< 

4 J 

i 


Repeat 


step 1 




2323 


0.5 


27 


C 2 8 


i 

5 J 


Repeat 


step 1 




3/09 


1.0 


28 


A 1 


• 

i 








1030 




29 


A 1 


• 

• 












30 


A 1 


i 

• 












31 


A 1 


i 

i 












32 


A 1 


i 

* 

i 












33 


C27 


i 

6 : 
« 


Repeat 


step 1 


O.K. 


1130 


0.3 






< 

7 ; 


Repeat 


debug runs 1 and 2 


o 

« 


1 150 


1.2 






• 

i 

i 

i 


( same 


r esu 1 t s exoec t eo ) 




3/09 









Remar k, s l 

The ALGOL comoiler diagnostics do not always allow a complete 
investigation of all errors. 
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WORKSHEET FOR DEBUGGING PHASE 
PROJECT » : 2 DEBUG Run A : a 

Begin of Debug Run (day/time) : 03/09/1500 

End of Debug Run (day/time) : 03/10/1200 

u of Debug Steps incl. in Debug Run: 6 CPU time for Debug run (sec): .61 
CPU time for necessary compiles (sec) : 23.45 
a) 3.67 b) 5.02 c) 5.20 d) a, 60 e) 4.96 f) g) 



Man hours for this Debug Run : 3.0 (including creoafat ion of deCuq run) 

MAN 1 ) 



STEP 


PROGRAM 


OBJECTIVE AND EXPECTED 


ACTUAL 


day jhoursjerror 


COMMENTS 


* 


PART 


RESULT 


RESULT 


TIME ’/STEP! 


! * 


ANO cooeo 














i 


ERROR TYPES 
















1 ) Record when 
















error 


1 


A 1 1 


Get error free compile. 


2 comp i 1 e 


3/0O 






occurs 




subrou- 

tines 




e r rors 


1500 


0.5 








except 






1730 




37 


C23 




logical 
p roc e- 
dures 










38 


Ai 


2 




Repeat step 1 


O.K. 


1730 








3 


I/O 


Check input for trivial 


number of 


1950 


0.5 










cases 


succes- 


2000 




39 


C26 (READ and 








sors pla- 








READ0N con- 








ced in 








fused) 








wrong 

field 










a 




Repeat steo 3 with trace 
and preceding initializa- 
tion of freelists as well 




2130 


0.5 










as initialization of list 
of path headers. 


wrong va- 


2200 




41 


A 1 






Check function "SON" for 


lue re- 














correct return values 


t urned 














(trace should match with 


from "Num 














design considerations) 


ber of 
Succes- 
sors" 










5 




Repeat step 4 


Mi ssina 


2320 


0.5 












link 


2340 




43 


C 9 


6 




Repeat step 4 


O.K. 


2350 


0.5 














3/10 
1 100 









Remarks: Functions and subroutines are first checked for trivial 
cases and Boundary conditions. 
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WORKSHEET FOR DEBUGGING PHASE 
PROJECT * : 2 DEBUG Run * : 5 

Begin of Debuq Run (day/time) : 03/10/1600 

End of Debug Run (day/time) : 03/10/2400 

H of Debug Steps incl. in Debug Run; 5 CPU time for Debug run (sec): 1,35 

CPU time for necessary compiles (sec) : 29.49 

a) 5.71 o) 5.94 c) 5.43 o) 5.79 e) 6.62 f) g) 

Man hours for this Debug Run : 5.0 (including preparation of debug run) 

MAN 1) 



STEP 


PROGRAM 


OBJECTIVE AND EXPECTED 


ACTUAL 


DAY 


! HOURS ; ERROR ! COMMENTS 


£ 


PART 


RESULT 


RESULT 


time:/step: a 


! AMO C0DE0 
















| ERROR TYPES 
















1 ) Record when 
















error 


1 


A 1 1 


Check for trivial incuts 




3/10 






occurs 




P r i m i - 


for all primitives ana 


1 error 


1530 


0.5 








t i ves 


utilities. 


in link- 












and 

utili- 

ties 




i ng 


1600 




45 


C27 


2 




Repeat step 1 


1 error 


1730 


1 .0 


47 


D9 


3 




Repeat steo 1 


3 proaram 


1930 


2.0 












e r r o r s 


1 Q 30 




46 


C21 (state- 








found 


2100 




49 


ments in re- 
verse order) 
D 1 2 (wrong 
stopping 
Condi t i on ) 










2130 


/ 


50 


C8 (faulty 
initial ization 

0 f 1 OOP ) 


4 




Repeat step 1 * 


2 program 


2200 


1.0 












errors 


2250 




51 


S 4 (error 
in correct- 
ing). 










2300 




52 


C24 


5 




Repeat step 1 


0 . K , 


3/10 

2330 


0.5 







Remarks: Procedures checked for trivial cases and boundary conditions. 
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WORKSHEET FOR DEBUGGING PHASE 
PROJECT * : 2 DEBUG Run » : 6 

Begin of Debug Run (aay/time) l 03/10/2230 
End of Debug Run (day/time) : 03/11/1700 

a of Debug Steos incl. in Debug Run: 5 CPU time for Debug run (sec): 2*14 

CPU time for necessary compiles (sec) : 26.3 
a) 5.14 b) 6.62 c) 7.38 a) 7.16 e) f) g) 



Man hours for this Oeoug 0 u n : 6.0 (including oreoaration of debug run) 

MAM 1) 



STEP! PROGRAM 
8 ! PART 

• 

• 


OBJECTIVE AMD EXPECTED 

RESULT 


ACTUAL 

RESULT 


OAT 

time; 


I HOURS I ERROR 
1/STEP! A 

i 


COMMENTS 
AND CODED 
EPROR TYPES 


1 

• 












1 ) Record when 


1 

1 












error 


1 ! A 1 1 


Get error free comoile of 


8 comoile 


3/10 


1 .0 




occurs 


1 

• 


a 1 1 o rog r am Darts. 


e r r o r s 


2300 








1 

1 






2310 




53 


A 1 


1 

• 










54 


A 1 


1 

1 










55 


Ai 


1 

• 










56 


A 1 


1 

1 










57 


A 1 


1 

t 










58 


C28 


» 

• 










5R 


C 9 


1 

1 

1 










60 


C R 


• 

2 ! 


Reoeat step 1 


O.K. 


2330 


in 

o 






« 

i 






3/10 








i 

• 

• 






2400 








• 

3 ! Loa i c a 1 


Check for trivial cases 




3/11 


3.0 






! proce- 


while analyzing one oath. 


1 error 


1400 








Idures 


Check writing of a oath. 


found 


1400 




61 


C27 (reset 


! and 


(all functions should oer- 










f o rgo t ten ) 


{ - r i t i nq 


form as des i qned ) 












J o f a 














loath 

i 














4 | 


Repeat steo 3 


2 e r ro r s 


1545 


1 .0 






i 

i 




found 


1620 




62 


CIO ( w r ong 


i 

i 












s t odd i ng c on - 


i 

i 












d i t i on ) 


i 

• 

i 






1630 




63 


C 3 0 


i 

5 | 


Peoeat steo 3 




1630 


0.5 






• 

• 




O.K . 


3/1 1 








• 

• 






1700 
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WORKSHEET FOR DEBUGGING PHASE 
PROJECT U : 2 DEBUG Run * : 7 

Begin of Debug Run (day/time) : 03/11/1700 

End of Debug Run (day/time) : 03/12/1600 

tt of Debug Steps incl. in Debug Run: 5 CPU time for Debug run (sec): 2.61 
CPU time for necessary compiles (sec) : 38.78 
a) 8.05 b) 7.81 c) 7.88 a) 7.82 e) 7,22 f) g) 



Man hours for this Debua Run : 7.0 (including preparation of debug run) 

MAN 1) 



STEP 

a 


PROGRAM 

DART 


OBJECTIVE AND EXPECTED 

RESULT 


ACTUAL 

RESULT 


OAT J HOURS ! EPRGP 

timej/step: a 

i 

i 


COMMENTS 
AND CODED 
ERROR TYPES 












i 

i 


1) Record w h e » 












t 

i 


error 


1 


Proce- 


Check analysis of oathes. 


2 program 


3/1 1 


o.5 : 


occurs 




dure 


(All pathes are implemen- 


errors 


1700 


• 

• 






"Set 


ted indoubly linked lists) 




1730 


! 6a 


D 1 2 (wrong 




oaths" 








• 

• 


stopping con- 












• 

i 


di t i on ) 










1 7 a5 


! 65 


Dll (wrong 












i 

• 

• 


inaex ing) 


2 




Reoeat steo 1 


1 error 


1605 


0.5 ! 










found 


1310 


: 06 


Dll ( same as 












i 

• 

i 


error 65 ) 


3 




Repeat steo l 


5 errors 


1320 


3.5 : 










found 




i 

• 












3/12 


* 

■ 












1300 


: 67 


03 ( f o rgo t ten 










1315 


! 63 


D3 update 










1320 


: o r 


D3 of a vari- 












1 

1 


able) 










1325 


: 70 


C29 










1 3ao 


: 7i 


9a (misinter- 












• 

i 


pretation of 












* 

• 

i 


error source) 


a 




Repeat steo 1 


1 error 


1 400 


1.5 ! 72 


09 








founa 




i 

• 

• 




5 




Reoeat steo 1 


O.K. 


1530 


o.5 : 












3/12 


* 

• 












1600 


i 

• 
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WORKSHEET FOR DEBUGGING PHASE 
PROJECT U : 2 DEBUG Run U : 0 

Begin of Debug Run (day/time) : 03/13/1600 

End of Debug Run (day/time) : 03/14/1600 

n of Debug Steos incl. in Debug Run: 3 CPU time for Debug run (sec): 1&2.75 



CPU time for necessary compiles (sec) J 


: 19.59 








a) 6.63 b) 6. a5 c) 6.51 a) 


e) 




f ) 


g) 


'^an hours for this Debug Run : 17.0 (including oreparation of debug run) 








WAN 


1 ) 


STEP ! PROGRAM ! OBJECTIVE AMO EXPECTEO 


ACTUAL 


DAY { HOURS { ERROR { COMMENTS 


n | PART ! RESULT 


RESULT 


T I V E ! 


{/STEP! 


! 4 ; AMO CODED 


1 1 

I t 






i 

i 


! { ERROR TYPES 


1 t 

1 * 








I 1 ) Record when 


1 I 

1 t 








{ error 


1 l 

t t 








{ occurs 


1 I All ! Check various small d i r e c - 


1 error 


3/13 




• 

i 


! ! ted graphs for boundary 


found 


1600 


8.0 


i 

i 


! {conditions. 


in "hat- 






i 

i 


i t 

i i 


ching” 


2 00 0 




73 { C 29 (wrong 


i i 

i i 

• i 








{branching) 

i 


i i 

2 ! {Repeat steo 1 


l error 


3/ia 


6.0 


i 

■ 

■ 


i i 

i i 


f cund 


0 70 0 




i 

i 


i t 

i i 


( some 


1 100 




7a {010 


i i 

i i 


oaths 






i 

i 


• < 

• i 


skipped 






■ 

i 


i i 

i t 


during 






■ 

■ 


i i 

i i 

i i 


analysis) 






i 

i 

i 


t i 

3 { {Reoeat steo 1 


O.K. 


1 300 


3.0 


i 

* 

i 


• i 

• i 




3/14 




t 

i 


i i 

i i 




1600 




■ 

■ 


Remarks: Error analysis during this deoug run was 


extremely 




difficult and time consuming for the following reasons: 



- v ost results were verified by desk checking. 

- The implementation of oaths was checked by 
examining the contents of allocated items of 
both freelists ( M 1,M2). 

- Some structures being tested nad a very large 
number of oaths ( > 2000), 

- The orogram contains a great number of rather complex 
algorithms, each of which had to be tested with 
resoect to its boundary conditions. 
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WORKSHEET FOR DEBUGGING PHASE 
PROJECT * : 2 DEBUG Run * : 9 

Begin of Debug Run (day/time) : 03/13/1100 

End of Debug Run ( day / t i-me ) : 03/15/2300 

U of Debug Steps inch in Debug Run: 1 CPU time for Deoug run (sec): 3.R2 

CPU time for necessary compiles (sec) : 7.52 
a) 7.52 b) c) c) e) f) g) 



Man hours for this Debug Run : 3.0 (including preparation of deouq run) 

MAN 1) 



STEP | PROGRAM 
* ! PART 
1 
• 


OBJECTIVE AMO EXPECTED 

RESULT 


ACTUAL 

RESULT 


DAY ! HOURS ! ERROR 
TIMES/STEP: a 

i 

1 


COMMENTS 
AMD CODED 
ERROR TYPES 


• 

t 








i 

• 


1) Record * h en 


1 

1 








i 

i 


error 


1 

1 i All 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 


Check internal error 
messages . 

(Error messages 5,o,7, 
11,12,13 snculd be o r i n - 
t ed ) 


O.K. 


3/14 

2345 

3/15 

2300 


i 

3.0 ! 

I 

i 

1 

I 

1 

1 

l 

1 

1 

1 


occurs 
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WORKSHEET for debugging phase 
PROJECT * : 2 DE8UG Run * : 10 

Begin of Debug Run (day/time) : 03/16/1500 

End of Debug Run (day/time) : 03/16/1600 

# of Debug Steps incl. in Debug Pun: 1 CPU time for Debug run (sec): 6.75 
CPU time for necessary comoiles (sec) : 7.73 
a) 7.73 b) c) a) e) f) g) 



Man hours for this Debug °un : 1.0 (including precaution of debug run) 

VAN 1 ) 



STEP ! PROGS AM 
S ! D ART 
1 
1 


08JECT1VE AMO EXPECTED ! ACTUAL 

RESULT ! RESULT 

1 
1 


DA Y « HOURS ! EPRQR ! COMMENTS 

time:/step: 2 ! add CODED 

; ; error types 


1 

t 


1 

) 




! ! 1 ) Wecon when 


1 

I 


1 

1 




! ! error 


1 

1 


1 

1 




! ! occurs 


1 JAM 


Chec< some user deoenaent ! 


3/16 


i.o; : 


1 

1 


er ror messages . ! 


1SQ0 


1 1 
1 1 


« 

« 


(error messages 10 ano 8 ! 0.<. 




1 1 
1 1 


1 

1 


shcu la be orintea) ! 




1 • 

1 1 


1 

1 


i 

i 


3/16 


t 1 

t 1 


1 

1 


• 

« 


loOO 


t • 

1 t 
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WORKSHEET FOR DEBUGGING PHASE 
PROJECT U : 2 DEBUG Run * : 11 

Begin of Debug Run (day/time) : 03/16/1600 

End of Debug Run (day/time) : 03/17/1200 

* of Debug Steos i nc 1 • in Debug Run: 2 CPU time for Debug run (sec): 6.45 
CPU time for necessary comoiles (sec) : 14.95 

a) 7.90 b) 7.05 c) ' a) e) f) g) 



Man hours for this Debug Run : 2.0 (including oreoaration of debug run) 

MAN 1) 



step 

3 


PROGRAM 

PART 


OBJECTIVE AND EXPECTED \ ACTUAL 

RESULT J RESULT 

i 

< 


0 A Y ! HOURS ! ERROR 
T IME ! /STEP ! * 

l 

1 


COMMENTS 
AND CODED 
ERROR TYPES 






i 

i 




i 

i 


1 ) Record wnen 






• 

i 




• 

■ 


error 






i 

• 




i 

i 


occurs 


1 


A) 1 


Chec^ error messaqe 3 9. {error 


3/lo 


1.0 ! 








! mes s aae 


loOO 


1 

1 








\ a 9 not 


1800 


: 75 


C 2 1 






! d r i n t ed 
1 




1 

1 

1 




2 




1 

Reoeat steo 1 ! 


3/17 


1.0 ! 








1 0.K, 


1 140 


• 

i 








i 

i 


3/17 


< 

■ 








• 

i 


1200 


i 

• 
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of 3 



ERROR LISTING 



PROJECT * : 2 



Begin of Project (day/time) 
End of Project (day/time) 
Man hours for total project 



03/01/1900 

03/17/2200 

125.0 



ERROR 

a 


PHASE 

in which 
ERROR was 
dis- 
covered 


PHASE 
in which 
ERROR was 
made 


ERROR 

TYPE 

(see ANNEX F) 


TIME 

spent to 
solve the 
ERROR 
( M an 
min.) 


a of OTHER 
STATEMENTS OR 
PARTS OF THE 
PROGRAM 
AFFECTED 


1 


Cod i ng 


Cod i ng 


C28 


5 




2 


Coding 


Coding 


C28 


5 




3 


Codi no 


Coding 


C23 


2 




a 


Cod i ng 


Coo i ng 


C28 


1 




5 


Oebuggi nq 


Coding 


A l 


5 




6 


Debugg i ng 


Coding 


C23 


5 




7 


Debugg i nq 


Coding 


A 1 


5 




9 


Debugg i ng 


Coding 


A 1 


10 




o 


Codi n g 


Des i gn 


09 


15 


whole algorithm affected. 


10 


Codi n g 


Codi ng 


C 1 2 


l 




1 1 


Coding 


Codi ng 


C 2 8 


1 




12 


Codi n g 


Coding 


C23 


1 




13 


Coding 


Codi ng 


C 1 6 


1 




iu 


Coding 


Codi ng 


Cl 


5 




15 


Coding 


Coding 


C 1 7 


2 




16 


Cod i ng 


Coding 


C 17 


2 




17 


Des i an 


Design 


09 


1 




18 


Coding 


Design 


D9 


30 


whole algorithm affected. 


19 


Coding 


Design 


01 1 


5 




20 


Codi ng 


Coding 


CIO 


5 




21 


0 ebugg i ng 


Coding 


C23 


5 




22 


Oebugq i ng 


Codi n g 


C 1 7 


15 




23 


Oebugg i ng 


Codi ng 


C 2 7 


5 




2a 


Debugg i ng 


Cod i ng 


A 1 


2 




25 


Debugg i nq 


Coding 


C 1 7 


5 




26 


Debuga i ng 


Coding 


A2 


10 




27 


Debugg i ng 


Coding 


C23 


10 




28 


Debugg i ng 


Codi ng 


C 1 7 


5 




29 


Debugg i ng 


Coding 


Cl 1 


5 




30 


Debugg i ng 


Coding 


A 1 


15 
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of 3 



ERROR LISTING 



PROJECT * : 2 



TIME 

soenf to 
solve the 
ERROR 
(Man 
on' n . ) 



31 


Oebugqi ng 


Cod i ng 


A 1 


5 


32 


Debugg i ng 


Coding 


A 1 


2 


33 


Oebugqi ng 


Coding 


C27 


5 


3a 


Coding 


Cod i nq 


Cl 1 


1 


35 


Coding 


Design 


DR 


15 


36 


Cod i ng 


Design 


D9 


15 


37 


Debuggi ng 


Cod i ng 


C23 


5 


38 


Debugg i ng 


Coding 


A 1 


5 


3R 


Debugq i ng 


Coding 


C26 


30 


u0 


Oebugq i nc 


Coding 


C20 


5 


a l 


Debugq i ng 


Coding 


At 


5 


42 


Coding 


Oes i gn 


09 


20 


43 


Oebugg i ng 


Oe s i gn 


D9 


10 


44 


Cod i ng 


Design 


015 


5 


45 


Debugg i ng 


Coding 


C27 


10 


46 


Cod i ng 


Design 


09 


5 


47 


Debugg i ng 


Des i qn 


09 


60 


48 


Debugg i ng 


Cod i ng 


C21 


10 


49 


Debugg i ng 


Design 


D 1 2 


5 


50 


Debugq i ng 


Coding 


C 8 


15 


51 


Debuga i ng 


Oebuga i nq 


94 


20 


52 


Oebugg i ng 


Coding 


C24 


5 


53 


Oebugg i ng 


Coding 


A 1 


5 


54 


Debuggi ng 


Cod i ng 


A 1 


2 


55 


Debugging 


Coding 


A 1 


2 


56 


Debugg i ng 


Coding 


A 1 


2 


57 


Debuggi ng 


Coding 


A 1 


2 


58 


Oebugg i ng 


Cod i ng 


C28 


5 


59 


Debuggi ng 


Coding 


C 9 


5 


60 


Debugg i ng 


Coding 


C 9 


5 



Begin of Project (day/time) I 03/01/1900 
End of Project (day/time) : 03/17/2200 
Man hours for total protect : 125.0 



ERROR! PHASE 


PHASE 


ERROR 


n J in which 


in which 


TYPE 


{ERROR was 


ERROR was 


(see ANNEX F) 


! o i s - 


made 




' covered 

i 

i 







n of OTHER 
STATEMENTS or 
PARTS OP THE 
PROGRAM 

AFFECTED 



whole algorithm affected, 
whole algorithm affected. 



whole algorithm affected. 



Ahol e subroutine affected. 
(3 changes necessary) 
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ERROR LISTING 



PROJECT n ; 2 



8egin of Project (day/time) 
End of Project (day/time) 
Man hours for total project 



03/01/1900 

03/17/2200 

125.0 



ERROR 


PHASE 


PHASE 


ERPCR 


TIME 


tt of OTHER 


4 


in which 


in which 


TYPE 


spent to 


STATEMENTS CR 




ERROR was 


ERROR was 


(see ANNEX F) 


solve the 


PARTS OF THE 




dis- 


nade 




ERROR 


PROGRAM 




covered 






( M an 


AFFECTED 










min,) 




oi : 


! Debugs i ng 


Coding 


C2 7 


30 




62 ! 


! Deougg i ng 


Coding 


CIO 


10 




63 ! 


! Debuggi ng 


C o d i n g 


C 29 


10 




6a | 


! Debugg i ng 


Design 


D 1 2 


10 




65 1 


! Debugq i na 


Design 


Dll 


5 




66 ! 


! Debugg i ng 


Design 


Dll 


15 




67 ! 


! Debuggi ng 


Design 


03 


10 




68 : 


! Debuggi ng 


De s i an 


03 


10 




69 j 


! Debugq i nq 


Design 


03 


5 




70 : 


! Deougg i ng 


C o d i n g 


C 3 0 


15 




71 ! 


! Debugg i ng 


Debugg i nq 


8a 


10 




72 | 


! Debugq i ng 


Design 


09 


120 




73 ! 


! Deougg i ng 


C o d i n g 


C 2 9 


180 




7a | 


! Debugg i ng 


Design 


010 


60 




75 ! 


! Debugg i nq 


Cod i ng 


C21 


30 
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ANNEX E2 



ERROR J 
* ! 



1 

2 

3 

a 



5 

6 

7 

8 
9 

10 



it 

12 

13 

la 

15 

16 

17 

18 

19 

20 
21 
22 
23 
2a 

25 

26 

27 

28 

29 

30 

31 

32 



Page 1 of 3 



ERROR LISTING (COMMENTS) 



DAY ! COMMENTS 

TIME! (EVIDENCE, THOUGHTS, WHY WAS THE ERROR MADE? 
| WHY AND HOW WAS THE ERROR DISCOVERED? 

! ERROR BLOCKING, etc.) 



03/06! 

t530!Errors 1,2,3 were discovered while reading previously 
{written sections of code. 

1590! 

1620 ! 

1790!Lack of concentration while punching cards, 

{(Main disadvantage while ounchinq cards is that punched 
Idata is not immediately seen after each key strode.) 

1815! same as a 

t 8 1 5 ! P rogr amme r did not cneck programming manual, (error could 
{have oeen avoided) 

1815!same as a 
03/07 ; 

1305! 

1305 ! 

03/06! 

1000!Errors 10-17 were detected Oecause programmer punched cards 

{himself. It is quite natural that he uses this time to review 
! h i s code . 

1005! 

1005! 

1010 ! 

1025 ! 

1030! 

1 030! 

1505! 

1630!Error found during desk test. 

2130! Error found while punching cards. 

2150J same as 1 9 

2200 ! 

2200 ! Func t i on name used as local variable. 

2200 ! 

2200 ! 

22 95 ! I nc omo 1 e t e correction of error 3 22. 

23 1 0 ! Manda t o ry declaration omitted while punching cards. 

2323!Right parentesi s omitted. 

03/09! 

1030! Lack of concentration while punching cards. 

! (Programmer was tired.) 



l 0 30 ! same 


as 


28 


1030! same 


as 


28 


1030! same 


as 


28 


1 030! same 


as 


28 



1 28 



ANNEX E2 Page 2 of 3 

ERROR LISTING (COMMENTS) 



ERROR 

n 


DAY 

TIME 


comments 

(EVIDENCE/ THOUGHTS/ A H Y WAS THE ERROR MADE? 
WHY AMD HOW WAS THE ERROR DISCOVERED? 

ERROR SLOCKING, etc.) 


33 


03/09 

1030 




3a 


1U40 




35 


1530 


Extreme conditions not carefully analyzed. 


36 


1615 


same as 35 


37 


1730 


Lack of concentration while coding. 


38 


1730 




39 


2000 


"READ" and "READ0N" confused. 


ao 


2030 


Using a local variaole name from different subroutine. 


a l 


2200 




42 


2230 




43 


2340 


Use o* w r o n q oarameter. 


44 


03/09 

1040 


Error found while punching cards. 


45 


03/10 

1600 




46 


1700 


Thinking of boundary conditions while punching cards. 


47 


1730 


Desk test was not made carefully enough, (one case 


48 


1930 


omitted) 

Two statements in reverse order. 


49 


2100 




50 


2130 


Error was discovered during Structured Walk Through. 


51 


2250 


Caused by changing code. 


52 


2300 




53 


2310 


Lack of concentration while ounchinq cards. 


54 


2310 


(see comment * 4) 
same as 53 


55 


2310 


same as 53 


56 


2310 


same as 53 


57 


2310 


same as 53 


58 


2310 




59 


2310 


Programmer did not check previous procedure declaration. 


60 


2310 


(faulty invocation) 
same as 59 


61 


03/1 1 
1400 


Forgotten reset of parameter. 


62 


1620 


Inapprooriate stopping condition in loop. 


63 


1630 


(confused with similar condition in a previuos problem) 
Wrong use of a subroutine. 


64 


1730 


Error found using a trace. 


65 


1745 


same as 64 


66 


1810 


Error found during desk test. 
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ERROR LISTING (COMMENTS) 



ERROR 

a 


DAY 

TIME 


COMMENTS 

(EVIDENCE/ THOUGHTS/ WHY WAS THE E p ROR v AOE? 
WHY AND HOW WAS THE ERROR DISCOVERED? 

ERROR BLOCKING/ etc,) 


67 


03/12 

1300 


Necessary uodate of a variable left out. 


68 


1315 


same as b7 


69 


1320 


same as b7 


70 


1325 


Boolean expression does not meet the needs of the 


71 


1 3ao 


algorithm, Er^or found by tracing. 

Mi si nterprtat ion of the error source. Error founa 


72 


1 a 00 


while repeating debugaing step. 

Different Penaviour of algor i t m for removal of 


73 


03/ la 
2000 


last path previous to the last one was not considered. 
Error found Dv d e s * test of all relevant suorcutines. 

Error found durinq desk test. 


7a 


03/15 
1 1 00 




75 


03/16 

1800 


Error found during desk test. 
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ANNEX F 



FINAL STATISTICS 



PROJECT » 2 



FINAL STATISTICS 

Project name : PATH ANALYSIS IN OIRECTED GRAPHS 
Short descrict ion: 

This program is designed to find all possible oaths of 
minimal length (i.e. if a directed orach contains I ooos no 
1 ooo may be traversed more than once in a row) through a 
di rec ted graoh . 

Inout : via ounch cares Output: via line orinter 
Quantitative measures: 

1 • * of source statements : 712 

2. Total man hours for oroject : 125.0 

3. M an hours soent in 

a ) Design : 31.0 

b ) Coding : 26.0 

c ) Oebugg i ng : 55.0 
d) Testing : 13.0 

4 . CPU time for comoiles: 206.85 sec. 

5. CPU time for debug runs: 139.46 sec. 

o. CPU time for test runs: 869.73 sec. 

7. a of test and debug runs: 13 

8. a of test and debug steos: 43 

9. 4 of errors found: 75 

10. Total man hours used to correct errors: 16.5 
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ANNEX F 



FINAL statistics 



T U of errors 



as 



ao 



35 



30 



25 



20 



15 



10 



* * 
* 



11. Error Detection : 

Mean time between error detections: 69 man min. 



★ * * 
* * 



5 



1 



} * * * * * * 

J * * * * 

| * * * * 

J * * ★ ★ * * * 

******* *★ 

J******* ** 



* * * * * 



* * * 






1 2 3 a S 6 7 8 9 1 0 1 1 12 13 

TIME BETWEEN ERROR DETECTIONS 

(measured from point in time of aetect ion of previous error) 



man hours 
+ --> of 

project time 
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ANNEX F 



FINAL STATISTICS 



12. Error Correction: 

Mean time to correct an error: 13.2 man min. 

U of 
Errors 
cor rec t ed 
T 



29 



25 



20 



15 



10 



★ * 
* * 
* * 
* * 
* * 
# * 
* * 



5 10 15 20 



25 30 oO 120 180 

man minutes 
of Project time 

TIME TO CORRECT ERRORS 

(measured in from ooint in time of detection) 
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ANNEX F 



FINAL statistics 



13. 


When 


e r r or s 


were 


found : 












a) 


n 


0 f 


e r ror s 


found 


d u r i n g 


design ohase: 


1 


= 


1.3 


X 


b) 


a 


o f 


er ror s 


f ound 


dur i ng 


design review: 


0 


z 


0.0 


X 


c) 


a 


o f 


e r r or s 


found 


du r i ng 


coding: 


21 


= 


28.0 


X 


d) 


n 


o f 


e r ro r s 


found 


d u r i n g 


debugging: 


53 


z 


70.7 


X 


e ) 


u 


of 


e r ror s 


found 


du r i ng 


w r i t i ng of 




















test orocedures: 


0 


= 


0.0 


X 


f) 


3 


of 


er ro r s 


found 


during 


test i ng : 


0 


= 


0.0 


X 



75 



14. 


When 


errors 


were 


made : 












a) 


u 


O f 


e r ro r s 


made 


du r i no 


design phase: 


20 


- 


26.7 


* 


b) 


n 


O f 


e r r o r s 


made 


during 


desicn review: 


0 


= 


0.0 


X 


c) 


3 


0 f 


errors 


"naOe 


du r i ng 


coding: 


53 




70.7 


V 


a) 


3 


0 f 


e r r o r s 


made 


du r i ng 


debuoa i ng : 


2 


= 


2.7 


x 


e) 


3 


of 


errors 


mane 


du r i ng 


writing of 




















test orocedures: 


0 


r 


0.0 


V 


f ) 


3 


o f 


er ror s 


made 


auri ng 


testing: 


0 


= 


0.0 


V 

/. 



75 
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ANNEX F 



FINAL STATISTICS 



1 5 ♦ TIME 


HISTORY GRAPHS 


: 










u of 
















E r ro r s 
















T 

i 
















i 

• 

% 












Error T y oe s : 


• 

i 












A : 


Clerical error 


i 

i 












3: 


Debugging error 


i 

i 












C: 


Coding error 


i 

i 












D: 


Des i gn error 


10 + 
















i 

• 












C 




i 

i 












C 




i 

i 


C 










C 




i 

i 


C 


c 








C 


D 


5 + 


C 


C A 








A 


3 


i 

• 


c 


A A 




C 




CA 


C 


i 

* 


A C 


C A 




c 


D 


3 A C 


D 


i 

• 


CCC c 


CCCC 


c 


A 


0 D 


CA CD 


D 


i : 


CC A A DCDD 


DC A C 


CD 


CADDCCDA CD 


D C D C 


* • • • * 


+- 


-* 

i 

+ 

i 

+ 

i 


> * — 


♦ - * 


- + - 


+-+-♦- 


♦ + - + - + - 


2 


2a 




60 






30 


112 



NUMBER OF ERRORS FOUND VS PROJECT TIME 



125 



* of 
Errors 



T 



5 + 

i 

1 + 



2 



★ * * 

* * * * 

2a 



♦ * 
* * * * 
** * * * * 
* * * * * * * 
-+-+-+•+ 



60 



★ 

* 

★ 

* 

* * 

* * 

* * * * 

* * * * * * ★ * * * 

************* * 

80 



* * 

♦ -♦ 



+ - + -♦-+• 
112 



NUMBER OF ERRORS CORRECTED VS PROJECT TIM£ 



125 



man 

hours 



man 

hours 
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ANNEX F 



ERROR CATEGORIES AND TYPES 



1. Design Errors 

The fol 1 wing types of errors apply to both categories 
"System Design Errors" ana "Program Design Errors": 

Dl : Communication Error 

02 : Design Negl iaence 

03 : Forgotten Cases or Steps 

04 : Timing Problems 

D5 : Errors in I/O Concepts 

D6 : Data Design Error 

07 : Initialization Error 

08 : Inadequate Checking 

D9 : Extreme Conditions Neglected 

DlO: Sequencing Error 

Oil: Index i ng Error 

012: Loop Control Errors 

D 1 3 : M i suse of Boolean Expression 

D 1 4 : Mathematical Error 

D15: Representation Error 

016: Misunderstanding of Problem Specifications 

017: Other Design Errors 

2 • Coding Errors 

Cl : Misunderstanding of Design 

C 2 : Nealigence 

C3 : 1/0 Format Error 

C 4 : Misplaced Data Declaration 

C5 : Multiple Data Declarations 

Cb : Missinq Data Declaration 

C7 : Inadeauate Data 

C8 : Initialization Error 

CR : Error in Parameter Passing 

CIO: Inadeauate or Forgotten Checking 

C 1 1 : Level P rob l ems 

C 1 2 : Missinq Declarations of Block Limits 

C 1 3 : Case selection error 

C 1 4 : GO TO Problems 

CIS: Comment Error 

C l 6 : Forgotten Delimiter 

C 1 7 : Inconsistency in Naming 

CIS: rfrong Use of Nested IF Statements 

C 1 9 : Indexing Error 

C20: Inconsistent Use of Variables or Data 

C21: Sequencing Error 

C 22 : Flag Usage Problems 

C2 3 : Syn t ax Error 

C24: Loop Control Error 

C 2 S : Incorrect Exit from Subroutines 

C2b: Language Usage Problems 
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ANNEX F 



ERROR CATEGORIES AND TYPES 



C27: Forgotten Statements 

C28: Reoresen t a t i on Error 

C2R: Control Sequence Error 

CiO: Incorrect Subroutine Usage 

C31: Other Coding Errors 



3. C 1 er i ca I Errors 



A 1 : Manual Error 

A 2 : Mental Error 

A3 : Procedural Errors 

A4 : Other Clerical Errors 



4 • Debugging Errors 

91 : I naoo roo r i a t e Use of Oeougqing Tools 

62 : Insufficient or Inaporoor i ate Selection 

of Test Cases or Test Data 
93 : Misinterpretation of Debugging Results 

8u : Misinterpretation of Error Source 

65 : Neqligence 

06 : Other Debugging Errors 

5. Testing Errors 

T1 : Inaaeauate Test Case(s) or Test Data 

T2 : Mi s i n teroret at i on of Test Results 

T3 : M i s i n t e rp r e t a t i on of Program Soec i f i c a t i on 

Tu : Negligence 

T5 : Other Testing Errors 
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ANNEX G 



Pace 1 of 21 



D I PEC TED GRAPH REPRESENTATION 

PROJECT a ; 3 



Program oart : CARt, C A R 2 , C 0 P 1 ,CQP 2 , SE T C AR 1 , SE T C AR 2 , 

SETCDR 1 , SETCDR 2 , FREE 1 , FREE 2 , SPACE, 
INITIALIZE all, SET NAME, A 003 R 0 THER, 
BROTHER, NAME OF, PATH ID, MA<*F, 

NUMBER Or SUCCESSORS, PREDECESSOR, 
SUCCESSOR, SET PATH ID, RUT %£<T P ATH- 
HEADER, NEXT PAT'-*, P°E 7 I 0 U 3 PATH, 

LINK PATH n A C * , LINhPATH, L I N * FCRaAWD, 
L I M K B A C '< , DUPLICATE, AOGNOOE, 
ALTERNATIVE <\* Ay, INITPAThLlST 



COMPLEXITY MEASURES: 



NUMBER OF STATEMENTS: 3-17 

NUMBER CF NCDS$: 2 

NUMBER Cr ARCS: i 

NUf*6cR CF PATHS: 1 

CYCLCMATIC NUMBER : V(G) = 




REACHA8IL ITY CF NCD E 5 : 
NClE 1 : I 

\CCc 2 : 1 



SUM: 2.CCCGC0 



- E "* r o r = AS 
N S 

Error = AS 



(in MjvaEP OF 
SUCCESSORS ) 
(in PUT NExT 
OATnr-EADER) 




NOTE : All S3 suoroutines Have Uie s a t e structure 



ANNEX G 



p a c e ? of 2 ! 



OIRECTEO G^apm RE?c£sEiMTATION 



PROJECT a : 2 

Proaran oart : ALLOCATE!/ ALLOCATED 



COMPLEXITY MEASURES: 



NUMBER Of STATEMENTS: 7 - B 

NUMBER CF NCC£S: 

NUMBER CF ARCS : 5 

NUMBER CF PATHS: 2 

CYCLE MAT IC NUMBER : 7(G) = 



REACHABILITY CF NCCESJ 



NGCE 

NCCE 

NGCE 

NGCE 



1 



SUM: 4.CCCCCC 

RE ACH A3 I L ITY INCSX 
C F OIRECTEC GRAFh: 



UCCCCCG 
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I locate c) 



ANNEX 5 p aae3of21 



DIPECTED GPAPm REPRESENTATION 

PROJECT a : 2 

Program part : EPPOP 



COMPLEX I T r MEASURES : 
NUM8EP OF STATEMENTS: oU 




i«o 



ANNEX G 



Pace 'A of 21 



DIRECTED GRAPH 



PROJECT # : 2 

Program cart : DIAGNOSE 



COMPLEXITY MEASURES: 



NUMBER OF STATEMENTS 

NUMBER C p i\CC£S: 
NUMBER CF ARCS : 

NU NB ER CF PATHS: 
CYCLCMAT IC NUMBER : 



: 32 



3 
9 

4 

V ( C- ) = 



3 



RE AC HA 3 I L ITY 
iNOCE 1 : 

NCCE 2 : 

NCCE 3 : 

NCCE 4 : 

NCCE 5 : 

NCCE t : 
NCCE 7 : 

NCCc a : 

SUM : 

PEACHA3 II ITY 

:f directed 



CF NODES 
1 
1 
2 

1 

2 
2 
4 
4 



17 .COCO 
IfNCcX 
GRAPH : 



2 . I2E0C0 



- l a i 



9E°9E'SE''lTiT l ON 



o 




0 



ANNEX G 



Pace 5 of 2 1 



(DIRECTED GPAPh REPRESENTATION 



PROJECT * : 2 

Program cart : SON 



COMPLEXITY MEASURES: 



NUMBER OF STATEMENTS: IS 
NLm v 6 5 R C f C C £ 3 : 
iN U . v 3 3 R Cr ARCS : 7 

MJN06R CF PATHS: 3 

CVCLC< V AT IC NUYcEF: V(G)= 2 



RE ACHA8 IlITY C c NCGES: 
i 
I 

1 
1 

2 

1 

2 



9.;03CCC 
REACrAE IL I7Y INDEX 
CF CIRECTEC GRAPH : 1.2657 14 



i\CCE 
VOCE 
NCCE 
\CCE 
N C Cc 
\CCE 
\CCE 



3 L, v i 




- 1«2 - 



fiNNfcX G 



Pace o o f 2 1 



OIPECTEO QWiPH REPRESENTATION 



PROJECT a : 2 



Program oarr : FIND ORIGINAL 



complex tty MEASURES : 



NUMBER OF STATEMENTS: It 

NUMBER CP NCC'cS: 6 

NUMBER CF ARCS : c 

NUMBER CF PATHS: 4 

CYCICMA7IC NUMBER : V<G) = 2 



REACHABILITY 


CF NCCES: 


NGCE 


1 : 


1 


NCCc 


2 : 


? 


NCCE 


3 ; 


1 


NGCE 


A : 


2 


NCCc 


c « 


2 


NO CE 


6 : 


2 


SUM : 




iC.COGOO 


S6ACHA6 I L I T Y 


I.NCEX 


CF CIREC 


TEC 


GRAPH : 



O 




- 1 a 3 - 



ANNEX G 



Pace 7 of 21 



DIRECTED GRAPH REPRESENT at ION 



PROJECT 3 : 2 

program oart : OUPLICATE CURRENT PATH 



COMPLEXITY MEASURES: 

JUMPER OF STATEMENTS: 15 

NUMBER CP NCCES: 5 

NUMBER CF ARCS : 5 

NUMBER CF PATHS: 3 

CYCLCMATIC NUMBER : V(C) = 



CF 



RE ACFA2 It ITY 

ncce i 
NCCE 2 
NCCE 3 
NCCE A 
NCCE 5 



SUM : 7.:COOOO 

REACHAg II ITY INDEX 
C c C IRECTEC GRAPH : 



NCCES: 

1 

1 

-5 

1 

2 



1.4CC00G 




- 1 44 



I 1AJ 



ANNE X G 



p ace P of 21 



DIRECTED GRAPH REPRESENTATION 



PROJECT a : 2 

Program oart : F I N 0 PATH 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: 12 
NUMBER CF NCCES: 7 

NUMBER CP ARCS : 7 

iNU >£ ER CF PATHS: 3 

CYCLC. M AT IC NUMBER: V ( G ) - 



REACHABILITY CF NCOES: 



NCCE 
NCCE 
NO Cc 
NClE 
NCCE 
NClE 
NCCE 



SUM : 



1C.CCCOO 



E AC HA 8 IL I TV INOcX 
F OIRECTEC GRAPH: 



1 . 42 65 7 1 




1 «5 



ANNEX Q 



Pane P o f l 



directed graph c eppesentat ion 



PROJECT a ; 2 

Program cart : REMOVE PATH 



COMPLEX ITT MEASURES: 



NUMBER OF STATEMENTS: uS 
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ANNEX G 



Pace 10 of 21 



DIRECTED GRAPH REPRESENTATION 



PROJECT a : 2 

Program part : END OF PATH 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: 18 

NUMBER CF NCCES: 9 

NUMBER CF ARCS : II 

NUf*BER CF PATHS: 9 

CYCLCMATIC NUMBER: V(G); 



REACHABILITY CF 



NOCE 

NCCE 

NOCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NOCE 



N C C £ 
1 
1 

1 

1 

2 

2 

S 

3 



IS : 



SUM: 25 • 0 3 000 

REACHABILITY INC6X 
CF DIRECTEC GRAPH: 2.777777 
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annex g 



Pa ae 11 of 21 



DIRECTED GRAPH REPRESENTATION 

PROJECT a : 2 

Proaram o a r t : INPUT DIRECTED G y A P n I ,m F 0 P m A X [ 0 N 
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A N N t X G 



Pace 12 21 



directed GRAPH representation 



PROJECT p : 2 

Proararr* oact : GET SUCCESSORS 



COMPLEX I T y MEASURES : 



NUMBER OF STATEMENTS: 34 

NUH6ER CF NCCES: 9 

NUV 0 ER CF ARCS : 9 

NU f»S EF CF PATHS: 5 

CYClCYATIC NUMBER: V(G) = 



REACHAeil IT Y CF NCCES: 



\CCE 

NCCE 

NCCE 

,\OCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 



SUH: 13 . C 00 JO 

REAChAE II ITY INCSX 
CF DIP6CTEC C-PAFH: 1.444444 
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ANNEX G 



Page 13 of 21 



directed graph representation 



PROJECT a : 2 

Program cart : OCCURS TWICE 



COMPLEX r TY MEASURES : 



N U M 8 E P OF STATEMENTS: 22_ 
N L . v 6 E R CF NCCSS: 11 

NUi v S E 5 CF ARCS : 14 

NU Y 9 E F CF PATHS: 23 

CYCLC V ATIC MJ M 3 E R : V(G) = 



RSAC1-A3ILI T Y CF NCCES: 
NCCE 
NO CE 
NCCE 
NC CE 
NCOS 
NC CE 
NC C E 
NC CE 
NC CE 
NC CE 
NQ CE 



6 

7 

a 

9 

1C 

11 



6 

7 

7 

14 

14 

2 a 



SUM: 92 • CCCCO 

R E AC r A 3 II 1 TY INC c X 
CF QIRECTEC GRAPH : 3 .45*t: 



45 



O 
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ANNEX G 



^ao? IN of 2 



DIRECTED GRAPH REPRESENTAT ION 



PROJECT - : 2 

Program n a r t : ONE y 



COMPLEXITY MEASURES: 



NUMREP OF STATEMENTS: i L 

♦NUMBER CF NCCES: 4 

MJi v BEk C r ARCS : 4 

iSUrScr CF PATHS: 2 

CVClC . v A T I C NUMBER: V(G)« 



REACHABILITY CF NCOcS: 
NO Cc I : I 

NCCe 2 : 2 

NCCE 3 : I 

NCCE 4 : 2 



SUY: 6 • C COO 30 

RE AC HAS IL ITY INCEX 
CF C IF EC TEC GRAPH: 1.5CC0CC 
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A M N £ X G 



Pace 15 o ^ 



DIRECTED GRAP* °£PkF5£!mT ATIGN 



PROJECT s : ?. 



P r o a r a r* o a r t : v A f C H [ >: G 

a ) 3 o t nones : 7 
fc) 3 0 f arcs : Q 
c) 3 c * 3 f 3t°-e^s: 1° 

j) 3 q * oa^s : * 

e ) J eacnaoi 1 i rv : * 

f) Cvc I era t i c nu-ne'*: 

* I'junre r ? f cat 
ana r<?ac"*^ci 1 i ' v 
ar® very ) area , 
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ANNEX G 



P a a e ! o o t 2 1 



DIRECTED GPAPh REPRESENTATION 

PROJECT * : 2 

Program o a r t* : CH£C<RACK 

a) 3 of nodes: 15 

b ) - of arcs : 1 o 

c ) 3 of statements: 5 n 

d ) 3 of oatns: * 

e ) Reachability: * 
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ANNEX G 



Pace 17 O' 2 1 



DIRECTED GRAPH R E D R E S E M T A T I G N 



PROJECT = : 2 

Program oart : SAVE tV E ,v 1 0 P Y 



COMPLEX I T r MEASURES : 



MUMPER OF STATE M E MTS: 2u 
NUMBER CF ^ Cl 5 S : 6 

NUMBER CF ARCS : 6 

(NUMBER G F r A T h $ : 2 

CVCLC MAT IC NUMBER: V ( G > = 



2 



R 6 >-C r A 8 I L I 
NOCc I 

[SCCE 2 

NCCS 3 

NCCc A 

NCCr 3 

NGCi e 



TV CF NGCcS: 
: i 

: i 

: 1 

: 1 

: 2 

: 2 



SUM: 3.CCCCGC 

REAChAE I L I T V TNCSX 
CF DIRECTED GRAFF : 1.323233 
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ANNEX G 



°aae IP o f 2 1 



DIRECTED G?AP-i PE°PEStNT A T ION 



PROJECT * : 2 

Program nart : LIST PATH 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: 2n 
MJNBEP CF NCCSS: 17 

NUM0ER CF ARCS : 19 

N U N 0 E R CF FATFS: 12 

CVCLCMATIC NUM6ER : V(G) = 



RE^CFAE 


IL 


NC Cc 


I 


NCCE 


2 


NCCE 


i 


NCCE 


4 


NCCE 


c 


NCCE 


6 


NCCE 


7 


NCCE 


a 


NCCE 


9 


NCCE 


LC 


NCCE 


II 


NCCE 


12 


NCCE 


12 


NCCE 


1 * 


NCCE 


15 


NOCE 


16 


NOCE 


17 



lr 



NCCS 

l 

I 

1 

1 

2 
2 
2 
2 
2 
2 

4 
2 

5 

a 

E 

a 



SU^ : 

REACFA8 II ITY 
CF C I R EC TEC C 



5C . COCCC 
INCcX 
RAFF : 



2 . S 4 1 1 7 6 




155 





ANNEX G 



P aae IP of 2 l 



DIRECTED GSAPm ^PPPESENTATION 



PROJECT 3 ; 2 

Program oart : LIST ALL PAT^$, ,vi A f N 



complexity VEASUPES: 



NUMBER OF STATEMENTS: 10 - 

NUMBER Cr NCCES: 

NUMBER CF ARCS : 4 

N U f* 6 c R CP PAThS: 2 

CYCLCYaTIC NUMBER: V ( G ) = 



1 1 



i 



REACHABILITY CF NCCES: 
NGCE L : I 

NCCc 2 : 2 

\nr? a : 1 



SUY: 6 • C COCQO 

RSAChAB IL TTY IMCcX 
CF DIP EC TEC GRAPH : 




I.5CCCC0 




ANNEX G 



Pag® 2 0 of 2 1 



DIRECTED GRAPH 



PROJECT ^ : 2 



Program Dart : AfiAL/ZE 



COMPLEXITY ^ASURES: 



NUMBER OF STATE'-’EMTS: 33 

NUMBER CF NCCES: 7 

NUMBER CF ARCS : 8 

NUMBER CF PATHS: *♦ 

CYCLCMATIC NUMBER : V(G) = 



REACHA2 IL ITY CF \CDS5: 
1 : 1 

2 : I 

2 : I 

4 : 2 

5 : 4 

6 : 2 

7 : a 



SUM: 15 . CO C 00 

RcAChAE II ITY INCc* 

OF OIRECTEC GRAPH: 



iSCCE 
NCCE 
\3C5 
NO CE 
NG l E 
NCCE 
NOCE 



2 . 1 A 2 5 5 7 



REPRESENTATION 




O 
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A N N £ < G 



P aae 2 l or 21 



DIRECTED GRAPH R E p R E S E u T A T I 0 M 

PROJECT ^ : 2 

Program oart : SET PATHS 
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ANNEX H 



Page 1 



of 2 



TEST PHASE DESCRIPTION 



Project A : 2 

Test run tt : 1 Including I Test Steo 

8egin of Test (day/time) : 03/17/1500 End of Test (day/time) : 03/17/1900 
CPU time for necessary comoiles (in sec,): 7,53 



a) 7,53 b) c) 


d) e) 


f ) g ) 




CPU time for TEST run (sec) : 25,29 






Man Hours for this Test 


run : 3,0 (including preoarat ion of 


tests) 












1 ) 




TEST! 08JECTVE 


EXPECTED 


ACTUAL {ERROR ! DA Y 


COMMENTS 


step; 


RESULT 


result: * {TIME 


AND 


* 

« 


(TOLERANCE) 


1 1 

1 « 


CODED ERROR 


• 

« 




• 1 

1 l 


TYPES 


» 

i 




* i 

• i 


l ) Record when 


i 

» 




i i 

i i 


error occurs, 


1 J Test oer f ormance 




: : 3/ i 7 




lof program for va- 




: :i5oo 




Jrious directed 




i i 

• i 




1 gr aoh s : 




• i 

« i 




! a ) di rected graoh 


3 valid oaths 


o.k. : : 




|9 nodes, 10 arcs 




i i 

i * 




!b)directed graon 


3 valid paths 


o.k. : ; 




J6 nodes, 7 arcs 




i i 

1 9 




J c ) di rec t ed graph 


<4 valid oaths 


o.k. : ; 




! 6 nodes , 7 arcs 




• i 

« i 




!d)di rected graph 


7 valid oat hs 


o.k. : ! 




! 22 nodes, 25 arcs 




i * 

t i 




! e )di rect ed graph 


11 valid paths 


o.k. ; ; 




! 22 nodes, 27 arcs 




« i 

i i 




{f)di rected graph 


no valid oath 


o.k. j • 




1 1 node , 1 arc 


warning should 


i i 

i i 




1 ( se 1 f 1 oop a t node 


be orinteo 


i * 

i * 




; i) 




i i 

i i 




! g) di rect ed graph 


2 valid oaths 


O.K. ! J 




J2 nodes, 2 arcs 




i i 

< * 




1 h ) di rect ed graph 


1 valid path 


o.k. ; ; 




!2 nodes, 1 arc 




: j3/i7 




• 




; :i9oo 




Remarks: Directed graphs submitted for testinq include a variety of 


boundary conditions and special cases which are considered 


to be difficult to examine. 
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ANNEX H 



Page 2 



of 2 



TEST PHASE DESCRIPTION 



Project * : 2 

Test run U : 1 Including l Test Steps 

Begin of Test (day/time) : 03/15/1200 End of Test (day/time) : 03/17/2200 

CPU time for necessary corroiles (in sec,): b.83 
a)6,83b) c) d) e) f) g) 

CPU time for TEST run (sec) : 8^4.44 

Man Hours for this Test run : 10.0 (including preparation of 

tests) 



1 ) 



test; objectve 

step: 

« 

* 

• 

» 


! EXPECTED 
! RESULT 
! (TOLERANCE) 


ACTUAL 

RESULT 


ERROR { DAY 

» : time 

1 
1 
1 
I 


COMMENTS 

AND 

CGDE0 ERROR 
tyres 


i 

• 






1 

» 


l ) Record *nen 


i 

• 






1 

1 


error occurs. 


l {Find limits of the 


! Program should 




{3/15 




| program using a 


{list all oaths 




! 1200 




{directed graon ha- 
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Remarks: Due to extensive deougqing previous to testing no errors 
have been found during the testing pnase. 

Test data for this run are chosen to indicate limitations 
of the program oecause of the finiteness of time and memory 
available. 



APPENDIX C 



PROJECT DESCRIPTION 

Project A : 3 

Project title: PATH ANALYSIS IN DIRECTED GRAPHS 

WITH RESPECT TO REACHABILITY OF NODES 

Programmer : HOFFMANN 

Programming Language : ALGOL 

Programming environment: I 8M/360 /b7 , OS/MV T , BA TCH 

Design notes : see ANNEX A 

Program listing : see ANNEX B 

Coding notes : see ANNEX C 

Debugging^ notes : see ANNEX D 

Error Listing : see ANNEX E 

Final statistics : see ANNEX F 

Graphical representation : see ANNEX G 

Test phase description: see ANNEX H 

Starting date: 1 MAR 77 Ending date: 17 MAR 77 

EXPERIMENT DESCRIPTION 
1. Project description! 

This program is an extension of project A 2 . The graph 

analyzing program as developed in project # 2 was modified 

to calculate the reachability of each node. A summary of 

the calculations is printed at the end of the analysis. All 

features described in project 3 2 will be unchanged. 



1 to 1 



Input: via Dunch cards 
Output: via line printer 

(For inout description and error messages see 
descriotion of oroject U 2.) 



! 



project 
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2. Programmer's background: 

a) Experience in programming: 



Oct 


1970 ~ May 


1971 


Programming courses 


May 


1971 - Aor i 1 


1972 


Module Programmer 


May 


1972 - June 


1979 


Work in Test and Simulation Depart 



ment at the 

NAVAL COMMAND AMD CONTROL SYSTEMS 
COMMAND (FEDERAL GERMAN NAVY) 

Testing of tactical real time systems 
March 1975 - Jan 1977 Student at the NAVAL POSTGRADUATE 

SCHOOL? Monterey? Computer Science 

b) Experience in testing: 

Two years of work in testing and simulation. 

c) Experience in the area of the given problem: None. 

d) Experience in the proaammina language being used: 
Experience over a period of 18 months in more than 10 

programming projects. (Total number of source statements 
produced during that time was more 9000.) 

3. Psychological factors: 

a) Did the programmer like the project? Yes. 

b) How does the programmer like the Programming 
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1 anguage? 

Favorite programming language. 

c) Was the programmer satisfied by the way the problem 
was specified? 

Only minor criticism. 

d) How did the programmer like the programming environ- 
ment? 

The facilities (study room/ card punch room) were not 
conducive to efficient orogramming because of restricted 
SDace/ bad lighting and noise. 

e) Other factors: 

The recording of the experiment's data during the pro- 
ject affected speed and concentration considerably. 



H . Comments on Documentation 

For the documentation of each software development phase a 
special documentation form has been developed. These forms 
are designed to provide a firm guideline for the experiment 
programmer to record all data of interest for subsequent 
error analysis. 

- Begin and end of each step was recorded with resoect to 
day and time. 

- Each error was recorded when it is discovered. The error 
was then identified by a unique error number (1,2,...). 
Furthermore the time of discovery and the error type (using 
error types listed in ANNEX F) were recorded. 

- If appropriate, comments about error discovery, reason why 
the error was made, etc. were documented in ANNEX E2. 

- For each error the phase in which the error was made, the 
phase in which the error was discovered and the time spent 
to correct the error was recorded in ANNEX El. 

- For each step in any one of the software development 
phases the day/time of begin and end was recorded. In addi- 
tion, the time (in man hours) for each step was recorded. 
This excludes the overhead used for documentation of the 
experiment data. 
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ANNEX A 



Page 1 of 1 



WORKSHEET FOR DESIGN PHASE AND DESIGN REVIEW PHASE OF PROJECT n 3 









J wan j 




STEP 


PROBLEM AND PLANNED SOLUTION 


ALTERNATE 


DAY IHOURSIERROR 


COMMENT 


u 




SOLUTIONS 


T IME ! /STEP 1 u 




1 


Analyze ways to compute the 




i i 

3/19! 3.0 ! 






reachability inoex for each 




1500! ! 





node • 

-In general the reachability 
index of a noae is incremented 
by l when the node is added to 
the current oath or out as the 
first node on an alternative 
oath. 

-The reachaoility of nodes 
which belong to a path wnich 
has to be removed has to oe 
dec remen t ed oy 1 uo to tie 
node wrier has been the last 
branch ooint on this oat* ex- 
cluaina the branch ooint 
i t se 1 f . 



3/19 

1800 



2 ! Too-Oown Design: 

!a) Define additional subrou- 
{ tines: 

J- Uodate Reachability 
! ( i nc rement by l ) 

5- Correct Reachability 
! (decrement by 1 for all ore- 
1 vious nodes uo to and exclu* 
! dina the last branch ooint) 

{ - List Reac h ab i 1 i t y 
! ( output o f resu Its) 

Jb) Define necessary chances 
1 within current version 

! of the analyzing program. 



3/20! 4.0 
1600 ! 



3/20 

2000 
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Annex 8 
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PROCECLFE SPACE! INTEGER VALUE N ) ; 
COMMENT URITE N BLANK LINES; 
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DIAGNOSE (TRUE, TRUE) 
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L CG I C /! L PROCEDURE OCCURS_TW I C E < INTEGER VALUE RESULT LAST; 

INTECER VALUE ID; 

LOGICAL VALUE RESULT TCP); 

COMMENT LOCKING BACKWARD ON THE SAME PATH BEGINNING AT ‘'LAST'' ThE 
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CGIC£L PROCEDURE MATCHING; 

OMRENT RETURN LOGICAL VALUE OF WHETHER OR NOT A STRING OF LENGTH 

GIVEN BY DISTANCE (GLOBAL) IS REPEATED TWICE PREVICUS OF AND 
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ROCECLPE UP CAT E_ REACHABILITY! INTEGER VALUE POINTER, N ) ; 

OPI-ENT UPDATE THE REACHABILITY OF THE ORIGINAL NCDE POINTED AT BY 
POINTER USING THE VALUE OF Nj 
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ANNEX C 



Page 1 of 1 



WORKSHEET FOR COOING phase OF PROJECT * : 3 
Seginning of Coding (day/time) : 3/20/2000 
End of Coding (oay/time) : 3/21/1200 
Man hours : 4.0 (including punching of cards) 



1 ) 



cot 

BEGIN 

OAY/TIME 


) INC 

ENO 

DAY/TIME 


PROGRAM PART 


ERROR J DAY 
* : time 

2 


COMMENT 


03/20/2000 


03/20/2200 


-Chanqes 
-Subr ou tines 
(new ) 






1) Record when error is 
detected. 


03/21/1000 


03/21/1200 


- p unchinq cards 


1 


1050 


C19 
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ANNEX 0 



Page 1 of 1 



WORKSHEET FOR DEBUGGING PHASE 
PROJECT ft : 3 DEBUG Run ft : 1 

Begin of Debug Run (day/time) : 03/21/1200 

End of Debug Run (day/time) : 03/21/1520 

4 of Debug Steps incl. in Debug Run: 5 CPU tine for Debuq run (sec): 41,13 
CPU time for necessary compiles (sec) : 29.98 
a) 6.03 b) 8.43 c) 7.65 a) 7,87 e) f) g) 



Man hours for this Debua Run : 3.0 (including oreoarat i on of debua run) 

MAN 1) 



STEP! PROGRAM 
s | PART 

• 

i 


OBJECTIVE AND EXPECTED 

RESULT 


ACTUAL 

RESULT 


day jhoursjerror 

T I W E ! /STEP : * 

i 


COMMENTS 
AND CODED 
ERROR TYPES 


1 


New 

subrou- 

tines 

and 

p r ogr am 
changes 


-Get error free compile 


1 comp i 1 e 
error 


3/21 

1215 


1.0 


2 


1 ) Record when 
error 
occurs 
C 6 

( f o rgo 1 1 en 
dec laration 
of variable) 


2 




-repeat step 1 


1 pro- 
gram 
error 


3/21 

1315 


0,5 


3 


C24 


3 




-repeat step 1 


O.K. 


3/21 
1 345 


0.3 






a 




-Chec* reachability cal- 
culation for trivial cases 


wrong 
branch- 
point 
cal cul a- 
t i on 


3/21 

1400 


0.7 


a 


C 2 1 


5 




-repeat step 4 


O.K. 

/ 


3/21 

1450 


0.5 














3/21 

1520 
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ANNEX E 



Page l 



o f 1 



ERROR LISTING 



PROJECT * : 3 

Begin of Project (day/time) 
End of Project (day/time) 
Man hours for total project 



03/19/1500 
03/23/2300 
3 3.0 



ERROR 

tf 


PHASE 
in which 
ERROR was 
dis- 
covered 


PHASE ! 
in which! 
ERROR was! 
made ! 


ERROR 

TYPE 

(see ANNEX F) 


TIME ! * of OTHER 

spent to ! STATEMENTS OR 
solve the! PARTS OF THE 
ERROR ! PROGRAM 

( M an { AFFECTED 

min.) | 


1 


Coding 


Cod i ng ! 


C 1 q 


io ! 


2 


Debugg i ng 


Cod i ng ! 


C 6 


5 ! 


3 


Debugg i ng 


Coding ! 


C24 


5 : 


a 


Oeougg i ng 


Coding ! 


C21 


20 ! 

• 

* 



r 



ERROR LISTING (COMMENTS) 



ERROR ! 

ft \ 


OAY 

TIME 


COMMENTS 

(EVIDENCE, THOUGHTS, WHY WAS THE ERROR MADE? 
WHY AND HOW WAS THE ERROR DISCOVERED? 

ERROR SLOCKING, etc.) 




03/21 




i : 


1115 


Checking code while punching cards. 


2 ! 


1215 


Found through comoiler diaanostics. 


3 ! 


1315 




4 ! 


1400 


Found bv interpreting results of debug run. 
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ANNEX F 



FINAL STATISTICS 



PROJECT * 3 



FINAL STATISTICS 

Project name : PATH ANALYSIS IN DIRECTED GRAPHS 

WITH RESPECT TO REACHABILITY OF NODES 



Short descriotion: 

This project is an extension of project * 2 . The graph 
analyzing oroqram developed in project * 2 is modified to 
calculate the reachaoilitv of each individual node. At the 
end of the analysis the reacnaoility index of each node is 
printed. All features of the program being chanaed are 
or eser vea . 

Inout: via punch cards Outout: via line printer 

Quantitative measures: 



1. 9 of source statements : 70 (including necessary changes) 

2. Total man hours for project : 33.0 

3. Man hours spent in 

a ) Design : 7.0 

b ) Cod i ng : 4.0 

c) Debugging : 3.0 

d) Testing : 19.0 

4. CPU time for compiles: 54.08 sec. 

5. CPU time for debug runs: 41.13 sec. 

6. CPU time for test runs: 391.48 sec. 



7. a of test and debug runs: 4 

8. 9 of test and debug steos: 8 

9. tt of errors found: 4 

10. Total man hours used to correct errors: 0.7 
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annex f 



FINAL STATISTICS 



II. Error Detect i on: 

Mean time between error detections: 43.3 man min. 

U of 
errors 
T 



5 ♦ 



1 * 



******************* 



1 5 



105 120 



135 



30 45 60 75 90 

TI^E BETWEEN ERROR DETECTIONS 

(measured from ooint in time of detection of orevious error) 
12. Error Correction: 

Mean time to correct an error: 10.0 man min. 



» of 
E r ro r s 
cor rec t ed 
r 

5 + 



> man min 
pro j ec t 



1 ! * * * 

1 5 10 15 20 25 30 35 40 

TIME TO CORRECT ERRORS 

(measured in from point in time of detection) 



man minutes 
of project time 



\ 



0 f 

1 me 
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ANNEX F 



FINAL STATISTICS 



13. 


When 


e r ror s 


were 


found: 












a ) 


* 


o f 


errors 


found 


during 


design oh ase : 


0 


s 


0.0 


z 


b) 


a 


o f 


errors 


found 


during 


design review: 


0 


s 


0.0 


z 


c) 


tt 


0 f 


e r ror s 


found 


dur i ng 


cooing: 


1 


= 


25.0 


X 


d) 


u 


0 f 


er ror s 


found 


during 


debugq i ng : 


3 


r 


75.0 


z 


e ) 


a 


0 f 


e r r or s 


found 


during 


writing of 




















test procedures: 


0 


= 


0.0 


z 


f ) 


tt 


o i 


e r r o r s 


found 


du r i ng 


testing: 


0 


r 


0.0 


z 



4 



14. 


When 


er rors 


were 


nade : 




a ) 


tt 


o f 


errors 


-nade 


during 


design phase: 


o) 


tt 


o f 


e r ror s 


made 


du r i nq 


design review 


c) 


It 


o f 


er r o r s 


nade 


during 


cod i ng : 


d) 


tt 


0 f 


e r ro r s 


•nade 


du r i ng 


deougg i ng : 


e ) 


a 


o f 


er ror s 


nade 


dur i ng 


w r i t i ng of 












test orocedures : 


f ) 


a 


0 f 


er rors 


nade 


du r i ng 


testing: 



0 = 0.0 *4 

0 = 0.0 % 

a = 100 z 

0 = 0.0 4 

0 = 0.0 
0 = 0.0 

a 
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ANNEX F 



FINAL STATISTICS 



1 5 . TIME HISTORY GRAPHS : 



tt of 
E r rors 
T 



Error T ypes : 

A: Clerical error 
0: Debugging error 
C: Coding error 

0: Design error 



It C C C C 

man 

12345 10 15 20 25 30 33 nour 

. . .DESIGN CODING. .DEBUGGING. .TESTING 

(software aevelooment erases) 

NUMBER OF ERRORS FOUND VS PROJECT TIME 



a of 
E r ro r s 

T 

i 

5t 



» 

1» c c c c 

12345 10 15 20 25 30 33 hour 

NUMBER OF ERRORS C0RRECTE0 VS PROJECT TIME 
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ANNEX F 



ERPOR CATEGORIES AND TYPES 



1. Design Errors 

The follwing tyoes of errors apply to both categories 
"System Design Errors" and "Program Design Errors": 

D1 : Communication Error 

D2 : Design Negligence 

D3 5 Forgotten Cases or Steps 

Da : Timing Problems 

05 : Errors in I/O Concents 

Db : Data Design Error 

D7 : Initialization Error 

D8 : Inadequate Checking 

09 : Extreme Conditions Neglected 

DlO: Sequencing Error 

Dll: Indexi ng Error 

012: Loop Control Errors 

D l 3 : Misuse of boolean Expression 

D19: Mathematical Error 

D15: Represen t at i on Error 

D16: w i s unde r s t and i ng of Problem Specifications 

017: Other Design Errors 

2 . Coding Errors 

Cl : Misunderstanding of Design 

C 2 : Negligence 

C3 : I/O Format Error 

C4 : Mi S oi ace <j Data Declaration 

C5 : Multiple Data Declarations 

C6 : M issing Data Declaration 

C7 : Inadequate Data 

C8 : Initialization Error 

C9 : Error in Parameter Passing 

CIO: Inadequate or Forgotten Checking 

C 1 1 : Level Prob 1 ems 

C12: Missing Declarations of Block limits 

C13: Case selection error 

C 1 a : GO TO Problems 

CIS: Comment Error 

Clb: Forgotten Delimiter 

C 1 7 : Inconsistency in Naming 

C18: Wrong Use of Nested IF Statements 

C19: Indexing Error 

C 2 0 : Inconsistent Use of Variables or Data 

C21: Sequencing Error 

C22: Flag Usage Problems 

C23: Syntax Error 

CEa; Loop Control Error 

C25: Incorrect Exit from Subroutines 

C2b: languaae Usage Problems 
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ANNEX F 



ERROR CATEGORIES AND TYPES 



C27: Forgotten Statements 

C28: Representation Error 

C29: Control Sequence Error 

C30: Incorrect Subroutine Usage 

C31: Other Coding Errors 



3. Clerical Errors 

A 1 : Manual Error 

A2 : Mental Error 

A3 : Procedural Errors 

A 4 ; Other Clerical Errors 



4. Debugging Errors 



01 : Inaooroori ate Use of Deougginq Tools 

82 : Insufficient or Inaporoor i ate Selection 

of Test Cases or Test Data 
63 : Misinterpretation of Deougginq Results 

04 : M) sinteroretat ion of Error Source 

0b : Neoligence 

06 : Other Debugging Errors 



5. Test i nq Errors 

T1 : Inaaeauate Test Case(s) or Test Data 

T 2 : Misinterpretation of Test Results 

T3 : Mi sinteroretat ion of Program Specification 

T u : Negligence 

T5 : Other Testing Errors 
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ANNEX G 



P aqe 



l 



o f 5 



DIRECTED graph REPRESENTATION 



PROJECT * ; 3 

Program oart : CORRECT REACHABILITY 



COMPLEXITY MEASURES: 



NUMBER OF STATEMENTS: 10 

\u ve 6s CF NCC E S : A 

NUMBER CF ARCS : A 

N U N 9 E s 2 r PATHS: 2 

CYCLE » v AT IC NUH86R: V(G)= 2 



REACHABILITY CF NC3ES: 



NGCS I : I 

NCCE 2 : 2 

NCC= 3 : I 

>\C Cc R J 2 



SUM : 



R £ AC HA S It iTY 
CF DIRECTED 



o.CCOOOO 



INC6X 
C-PAPH : 



1.5CCOOO 
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ANNEX G 



DIRECTED Graph REPRESENTATION 



PROJECT s : 3 

Program cart : UPDATE REACHABILITY 



COMPLEXITY MEASURES: 



NUMBER 


OF 


STATEMENTS 


: 3 


(NUMBER 


CF 


NC C c S : 


2 


NU>6 ER 


CF 


A R C c : 


L 


.NU cR 


CF 


PATHS : 


1 


CYCLCMATIC 


Mj P 3 E F : 


V ( G ) = 



1 



REACHABILITY CF NCDES: 
N CCE I : I 

NCCE 2 : 1 



SU^: 2.CCOOOO 

REACHAB IL I T Y INDEX 
CF DIRECTED GFAFH: 



1.CC3JOO 




Page 2 o * 3 
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ANNEX G 



?aae 3 of 3 



DIRECTED GRAPH REPRESENTATION 



PROJECT * : 3 

Program part : LIST REACHABILITY 



COMPLEX I Ty MEASURES : 



NUMBER OF STATEMENTS: 13 

N U H B 6 R CF i\ C C £ S : 4 

NUMBER CF ARCS : 4 

NUYBEP C F PATHS: 2 

CYCICVATIC INUH0EP: V(C) = 2 



REACHABILITY CF SCQcS: 
NQCE I : l 

NCCE 2 : £ 



NCCE 4 



* SUV: 

R£ ACHAP IL ITY 

CF DIRECTED 



c.CCOCCC 
INDEX 
GRAPH : 



l.SCCJGO 
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ANNEX H 



Page 1 



of 3 



TEST PHASE DESCRIPTION 



Project * : 3 

Test run U : 1 Including 1 Test Step 

Begin of Test (day/time) : 03/22/1000 End of Test (day/time) : 03/22/1800 

CPU time for necessary comoiles (in sec*): 8.33 

a) 8.33 b) c) d) e) f) g) 

CPU time for TEST run (sec) : 80. 7u 

Man Hours for this Test run : 8.0 (including preparation of 
tests) 



1 ) 



TEST 

STEP 


CBJECTVE 


EXPECTED 

RESULT 

(TOLERANCE) 


actualjerror jday 

RESULT! u {TIME 

i i 

i i 

i i 

i i 


COMMENTS 

AND 

CODED ERROR 
TYPES 










i 

i 


1 ) Record when 










i 

i 


error occurs . 


1 


Compare resul t s 


P rog ram perfor- 


O.K. 


! 3/22 






of previous test 


mance shou Id not 




! 1000 






runs with results 


be ch anged with 




i 

i 






o f modi tied vers- 


respect to oath 




i 

i 






ion of the program 


analysis as being 




i 

i 






using same input 


specified in pro- 




• 

• 






data. 


ject * 2. 




! 

1 








Additional out- 




1 

1 








put (such as cy- 




1 

1 








c 1 oma t i C number 




1 

1 








and r eachab i 1 i t y 




1 

• 








index) must be 




1 

1 








verified by desk 




1 

1 








checking. 




1 

1 












I 3/22 












! 1800 
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Page 2 



of 3 



TEST PHASE DESCRIPTION 



Project # : 3 

Test run # : 2 Including 1 Test Steo 

Begin of Test (day/time) : 03/22/1800 End of Test (day/time) : 03/23/1800 

CPU time for necessary compiles (in sec,): 7.90 

a) 7.90 b) c) d) e) f) g) 

CPU time for TEST run (sec) : 13^.00 

Man Hours for this Test run : 8.0 (including preparation of 
tests) 



TESTi 0BJECTVE 

STEP! 

i 

i 

i 

• 


EXPECTED 

RESULT 

(TOLERANCE) 


i ) 

4C TUAL ! ERROR 

result: « 

1 

1 

t 

1 


DAY 

TIME 


COMMENTS 

AND 

CODED ERROR 
TYPES 


* 

i 




1 

■ 




1 ) Record w 


i 

i 




1 

1 




error oc< 


1 ICompare hang cal- 


In cases of dis- 


o.k. : 


3/22 




culated resu Its of 


c reoanc i es pro- 


i 

t 


1800 




I N IDS subroutines 


gram should pro- 


i 

t 






| o f modu 1 e 1 w i t h 


vide the cc r rec t 


i 

i 






I p rog r am resul t s . 


answer ot herw i se 


i 

i 






! Chec k for disc re- 


results must 


i 

i 






! pane i es (if any ) . 


match. In case of 


i 

i 






i 

i 


d i sc repanc i es re- 


i 

• 






• 

i 


sults must be 


• 

t 






i 

• 


verified by desk 


i 

• 


3/23 




i 

i 

i 

i 


checking. 


i 

i 

i 

i 


1800 
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ANNEX H 



Page 3 



of 3 



TEST PHASE DESCRIPTION 



Project 2 : 3 

Test run U : 3 Including 1 Test Step 

Begin of Test (day/time) : 03/23/2000 End of Test (day/time) : 03/23/2300 

CPU time for necessary compiles (in sec.): 7.87 

a) 7.87 b) c) d) e) f) g) 

CPU time for TEST run (sec) : 171.74 

Man Hours for this Test run : 3.0 (including oreoaration o* 
tests) 



1 ) 



TEST 

STEP 


OBJECTVE 


EXPECTED 

RESULT 

(TOLERANCE) 


ACTUAL ! ERROR ! 0 A r 

RESULT! » I TIME 
1 1 

1 1 


COMMENTS 

AND 

CODED ERROR 
TYPES 








I • 

1 1 


1 ) Record when 








1 I 

1 • 


error occurs . 


1 


Comoare hand cal- 


In cases of dis- 


O.K. J '3/23 






culated results of 


crepancies pro- 


: 12000 






NTDS sub rout i nes 


gram should oro- 


t 1 

1 < 






o f modu 1 e 2 with 


vide the correct 


1 1 
1 1 






Drogram resul ts. 


answer o t he rw i se 


t I 

1 1 






Check for d i sc re- 


resul t s mus t 


I • 

1 1 






panc i e s (if any ) . 


match. In case of 


1 • 

( 1 








a i sc reoanc i es re- 


1 • 

1 1 








sults must be 


( 1 

< 1 








ve r i f i ed by de s k 


; 13/23 








checking. 


1 12300 

i i 

« i 
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APPENDIX D 



PROJECT DESCRIPTION 

Project U : 4 

Project title: DATA RETRIEVAL SYSTEM 

Programmer : HOFFMANN 
Programming Language : ALGOL 

Programming environment: 1 8M/ 360 /67 , OS /M V T r B A T CH 

and TIME SHARING (CP/CMS) 

Design notes : see ANNEX A 

Program listing : see ANNEX B 

Coding notes : see ANNEX C 

Debugging notes : see ANNEX D 

Error Listing : see ANNEX E 

Final statistics : see ANNEX F 

Graphical representation : see ANNEX G 

Test phase description: see ANNEX H 

Starting date: 19 APRIL 77 Ending date: 3 MAY 77 

EXPERIMENT DESCRIPTION 
1. Project description: 

A, General Description 

The program is designed for usage under CP/CMS. It 
expects an input file labeled "DBASE INPUT" to contain data 
base information of the format described in section B. 

During initialization all data base information is read 
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into program memory. After this the user may operate upon 
the data using functions as described in section C. All 
functions are input via terminal. 

Most of the terminology being used throughout the pro- 
ject is non standard. Therefore the following explanation 
is provided for better understanding: 

a) DATA BASE MEMBER: see description under B. 1. 

b) ATTRIBUTE VALUE PAIR: see description under 8. 2. 

c) MEMBER ID: The program assigns a MEMBER ID to each 
data base member deoending on the input sequence. 
MEMBER IDs are integers (1,2,3 ...). The MEMBER ID can- 
not be changed by the user, however, it may be used to 
reference any particular data base member. 

d) ATTRIBUTE ID: The program assigns an ATTRIBUTE ID 
to each new and distinct ATTRIBUTE NAME being entered 
during the initial incut of the data base members. 
ATTRIBUTE IDs are integers (1,2, ...64) depending also on 
the input sequence. Each ATTRIBUTE ID is associated 
which a unique ATTRIBUTE NAME and provides a way of 
referencing any particular ATTRIBUTE NAME (see section C 
"LISTA") . 

e) ATTRIBUTE NAME: An ATTRIBUTE NAME is a character 
string of length 1-64 preceding the M : H in the input of 
an ATTRIBUTE VALUE PAIR (see B. 2.). 

f) ATTRIBUTE VALUE: An ATTRIBUTE VALUE is a character 
string following the " : " in the definition of an 
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ATTRIBUTE VALE PAIR (see B. 2*). 

g) KEY ATTRIBUTE: The KEY ATTRIBUTE refers to the 
second distinct attribute name being input. (The second 
attribute name has been chosen because the main usage of 
the proqram will have data base members which are iden- 
tified uniquely by their second attribute value pair.) 
Therefore the ID of the KEY ATTRIBUTE is initially 2 , 
however, this may be changed by the user to any other 
ATTRIBUTE ID by using the function "KEY" (see section 
C) . 

The KEY ATTRIBUTE is not always the M key attribute M in 
the sense being used in most data bases, because it does 
not necessarily allow a unique identification of all 
data base members. (Instead the MEMBER ID can be viewed 
as internal key which always allows a unique identifica- 
tion of each individual data base member*) 

h) CONTROL: If a data base member is taken into con- 
trol by using the function " CONTROL* 1 an internal refer- 
ence is set by the program which allows subsequent exam- 
ination of ATTRIBUTE VALUE PAIRS of this Particular 
member by using functions "FA" and "ATTR", 

\ 
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B. InDUt Format 



1* Description of a Data Base Member: 

A data base member is defined by 1 - 64 different 
attribute value pairs followed by the termination symbol 
"#"• After the last data base member an additional ter- 
mination symbol has to be inserted* 

2 . Description of Attribute Value Pairs: 

Each attribute value pair has the following format: 

<attribute name>:<attribute v a 1 ue> a 
where "attribute name" is a character string of length 1 
- 63 not including and "attribute value" is one of 

the following types: 

a) <s ingle discrete va1ue> 

A single discrete value is a character string of 
length 0 - n > where n is limited by the amount 

of memory available (see section D). 

A character string of lenqth 0 will be replaced 
internally by " * * * " and specifies an unknown 
va 1 ue . 

b) <$<multiple discrete val ues>> 

Multiple discrete values are any number of sin- 
gle discrete values separated by The total 

string length of all values including separating 
" / " may not exceed 64. 
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c) <5<range va1ue>> 

A range value is a string of length 6 - 64 of 

the format : 

<1ower limit> TO <uoper 1 i m i t > 
where ’’lower limit" and "upper limit" are in- 
teger values and <1 ower 1 i m i t > <- <upper 1 i m i t > • 

Examo 1 es : 

a) Attribute value pair with single discrete 
value: 

NAME:SMITH# 

b) Attribute value pair with multiple dicrete 
values: 

CHECKING ACCOUNT : $804020 *50033# 

c) Attribute value pair with range value: 

MONTHLY PAYMENT S : 3550 TO 7254 

3, Number of data base members: 

The data base may contain 1 -150 members depending 
upon the amount of memory being used for the storage of 
values (see section 0), 

4. Termination of input: 

In order to terminate the input of data base infor- 
mation an additional termination symbol ("4") has to be 
inserted after the the termination symbol of the last 
data base member. 
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5. Reserved characters: 

All characters other than " : " , " , " , " # " , " $ " , and 

"3" may be used without restriction. 

6 . Sample i npu t : 

NAME : ME YER # FIRST NAME : JOE# AGE:57*3 NAME:SMITHS 
FIRST NAME : M IKE« AG E: SS NAME : NEWMAN u 

FIRST NAME : M ARY ANNS AGE: 3 

NOTE: 

a) between attribute value pairs any number of blank 
characters (0,1,2,...) will be ignored by the program. 

b) Any number of blank characters preceding a value will 
be ignored. (In the example the age of the last data 
base member will be stored as "34".) 

c) Any number of blank characters greater than one 
within a value or attribute name will be reduced to a 
single blank. (In the examole the first name of the last 
data base member will be stored as "MARY ANN"and the 
attribute name of the second member will be stored as 
"FIRST NAME" although it was input as "FIRST NAME" .) 
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C. Function Commands: 



1. Functions without input parameters: 



INPUT 


FUNCTION DESCRIPTION 


LISTA 


All attribute names are listed togetner with their 
ATTRIBUTE ID. (For usage of ATTRIBUTE ID see 
functions " A T T R " and "KEY".) 


LISTC 


All available commands are listed. 


LISTDBASE 


All data base members are listed printing all of 
their attribute value pairs together with MEMBER 
ID of each data base member. (For usage of MEMBER 
ID see functions "CONTROL" and "LISTM".) 


LISTALL 


All data base members are listed printing only 
their KEY ATTRIBUTE and its value preceded by the 
MEMBER ID. (For usage of MEMBER ID see functions 
"CONTROL' 1 and "LISTM" . The KEY ATTRIBUTE is the 
second attribute of the first data base member 
by default and may be changed by using the 
function "KEY”.) 


SWITCH 


This allows the user to direct the subsequent 
output from terminal to a file labeled "DBASE 
OUTPUT" or or vice versa. To change the output 
from terminal to file might be appropriate if 
large output is expected. The contents of the 
outout file may be obtained using the CMS command 
"OFFLINE PRINTCC DBASE OUTPUT". 


& 


Termination of the program. 
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2. Functions which require only one input parameter 



INPUT OF 
FUNCTION 


parameter 


FUNCTION DESCRIPTION 


LISTM 


<MEMBER I D> 
must be integer 


All attribute value Dai rs of data base 
member specified by MEM8ER ID are listed. 


CONTROL 


<m£MBER id> 

must be i nteger 


Program takes data base member soecified 
by MEMBER ID into control. (For usage of 
this function see functions "FA" and 
" A T TR " . ) 


AT TR 


ATTRIBUTE I D> 
must be i nt eger 


List value of attribute specified by 
ATTRIBUTE ID of data base member in 
control. (To select proper ATTRIBUTE ID 
see function "LISTA". To take a member 
into control see function "CONTROL".) 


FA 


ATTRIBUTE 

NAME> 

must be a cha- 
racter string 
of length 1-63 
f o 1 lowed by H : " 


List value of attribute soecified by 
ATTRIBUTE NAME of member in control. 

(To take a member into contol see 
function " CONTROL* 1 • To select an existing 
ATTRIBUTE NAME see function "LISTA".) 


KEY 


ATTRIBUTE ID> 
must be i nteger 


Change key attribute of data base to 
attibute specified by ATTRIBUTE ID. 

(For usaqe of key attribute see functions 
"LISTALL" and "FIND ID". To select an 
ATTRIBUTE ID see function "LISTA".) 



Note: 
i npu t 
input 



If parameter is required to oe an integer any other 
will force the program to terminate. Other wrong 
will cause an appropriate error message. 



3. FIND and FIND ID: 

a) Function Descrition: 

These two functions allow the user to find all data case 
members which satisfy 1 thru 4 conditions specified by 
subsequent input. (By a condition is meant an attribute 
value Pair. A member satisfies the condition only if it 
owns this particular attribute value pair or if it owns 
the attribute without a value specified. The latter 
case will be indicated by printing "***£" as attribute 
value in the output following a query.) After the condi- 
tions have been input the program tests all data base 
members as to whether they satisfy all conditions. Only 
those members which satisfy all conditions of the query 
are listed. Using "FIND" all of their attribute value 
pairs are printed whereas using "FIND ID" only the key 
attribute and its value is printed. (To change the key 
attribute of the data base see function "KEY".) If no 
conditions are entered all data base members will be 
listed. 



b) Input of Conditions: 



CONDITIONS 1-4: 



Input of ATTRIBUTE NAME> ! 



ATTRIBUTE NAME can be any 
string of length 1-63 
followed by (it is 

necessary that the 
ATTRIBUTE NAME matches with 
one of the existing 
ATTRIBUTE NAMES) 

If an ATTRIBUTE NAME cannot 
be found an error message 
is printed and the user can 
repeat the i nput . 



Input of <attribute value> 



Within queries attribute values are 
restricted to be 1-64 characters. (If 
the value stored is longer than 64 
characters the comparison is only 
carried out up to the last character 
of the query value. ) 

Since all values are stored without 
preceding blank characters the query 
value should not have blank characters 
in front of the value to be tested for. 



c) Termination of Query Input: 



User i nput s : 
Q 



d) Sample Input for FIND and FIND ID: 
FIND 
NAME : 

SMITH 

AGE: 
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22 

Q 



FIND ID 
AGE: 

104 

Q 

NOTE: 

1. All input for all functions must not be preceded by 
blank characters, 

2, Each input has to Pe on a separate line. The input is 
entered by hitting the carriage return key. 

D. Limits of the Program: 

The total number of different attribute names may not 
exceed 64, Assuming 100 data base members having 64 attri- 
butes each, the overhead used for implementation (65 items 
for each data base member) would reduce the amount allocat- 
able items of M2 (20,000) to 13,500 items. Each item of M2 
can be used to store 1-4 characters. The number of remain- 
ing items divided by the number of data base members gives 
the average number of items available for each data base 
member ( approximately 135). Thus the total amount of 
remaining storage locations allow approximately 54,000 char- 
acters for values which means that the average string length 
of values will be less than 8. The program assumes that at 
least two different ATTRIBUTE NAMEs are entered. If this is 
not true the user has to chanae the KEY ATTRIBUTE to ” 1 ” • 

E . Error Messages : 

All error messages are self explanatory. Error #1 and 
#2 will cause the program to terminate. This will be the 
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case when all memory locations are exhausted (error #1) dur- 
ing the input of data base information or more than 1000 
members are specified (error u 2). 

Error #3 indicates that an attribute name is longer than 
63 characters. (In this case the problem might be solved by 
examining the input file.) 
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2. Programmer's background: 

a) Experience in programming: 



Oct 


1970 - May 


197 1 


Programming courses 


May 


1971 - April 


1972 


Module Programmer 


May 


1972 - June 


1979 


Work in Test and Simulation Depart- 



ment at the 

NAVAL COMMAND AND CONTROL SYSTEMS 
COMMAND (FEDERAL GERMAN NAVY) 

Testing of tactical real time systems 
March 1975 - Jan 1977 Student at the NAVAL POSTGRADUATE 

SCHOOL/ Monterey/ Computer Science 

b) Experience in testing: 

Two years of work in testing and simulation. 

c) Experience in the area of the given oroblem: 

Usage of similar aata structures in previous programming 

projects. 

d) Experience in the oroaamming language being used: 
Experience over a period of 18 months in more than 10 

programming projects. (Total number of source statements 
produced during that time was more 5000.) 

3. Psychological factors: 

a) Did the programmer like the project? Yes. 
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b) How does the programmer like the programming 
1 anguage? 

Favorite programming language. 

c) Was the programmer satisfied by the way the problem 
was specified? 

Only minor criticism. 

d) How aid the programmer like the programming environ- 
ment ? 

The facilities (study room, card punch room) were not 
conducive to efficient programming because of restricted 
space, bad lighting and noise. 

e) Other factors: 

The recording of the experiment *s data during the pro- 
ject affected speed and concentration considerably. 
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4* Comments on Documentation 

For the documentation of each software development phase a 
special documentation form has been developed* These forms 
are designed to provide a firm guideline for the experiment 
programmer to recorc all data of interest for subsequent 
error ana lysis* 

- Begin and end of each step was recorded with respect to 
day and time. 

Each error was recorded when it is discovered* The error 
was then identified by a unique error number (1*2*...). 
Furthermore the time of discovery and the error type (using 
error types listed in ANNEX F) were recorded* 

• If aopropri ate? comments about error discovery, reason why 
the error was made, etc* were documented in ANNEX E2. 

- For each error the chase in which the error was made, the 
phase in which the error was discovered and the time spent 
to correct the error was recorded in ANNEX El* 

- For each steo in any one of the software development 
phases the day/time of begin and end was recorded* In addi- 
tion, the time (in man hours) for each step was recorded. 
This excludes the overhead used for documentation of the 
experi men t data. 
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ANNEX A 



Page 1 of 3 



WORKSHEET FOR DESIGN PHASE ANO DESIGN REVIEW PHASE OF PROJECT * 4 



STEP 

* 



PROBLEM AND PLANNEO SOLUTION 



alternate 

SOLUTIONS 



J J MAN J 

jday jhoursierror 
jtimej/step: u 



COMMENT 



Analysis of reauirements of 
project. 

-Since almost all expected 
values are of different length 
usage of linked lists for sto- 
raae of all attribute values 
seems to be appropriate. 

-Study of I/O caoabilities un- 
der CP/CMS. 

-View of data base members: 
Each data base member has a 
uniaue identifier (member »). 
For each member max. 6 a diffe- 
rent attribute value pairs can 
be defined by inout. M aximum 
string length for any attri- 
bute is o4, The total number 
of different attributes in the 
data case may also not exceed 
6U (arbitrarily chosen limit). 



Storage in fixed 
l enat h s string 
arrays would 
waste too much 
memory. 



a) Let number of 
different attri- 
butes be an in- 
put oaramet er . 

b) Use a hash 
table to store 
attributes. This 
mou 1 a make 
string lenqths 
of attributes 
independent of 
the array s i ze 
of the attribute 
table. 



a/19 

1400 



4/19 

1900 



5.0 



2 



Define data structures: 
-Attributes are identified by 
a character string of length 
bu and stored in a string 
array (ATTRIBUTE). 

Each attribute can be uniauely 
identified by its position 
within the attribute array 
(ATTRIBUTE ID). 

-Values of attributes are 
stored in a linked list struc- 
ture using a BITS ARRAY (M2). 
(Length of M2 (124,000 bytes) 
limits the amount of informa- 
tion to be stored in the data 
base . ) 



4/191 4.0 

1900 1 



4/1 9 
0900 



P r i m i t i ves 
for dat a 
s t r uc t u r es 
could be 
copiedf rom 
project 
42 . 



Remarks: see cage la 
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ANNEX A 



Page la of 3 



Remarks : 



a) Linked list of oata base members: 



1 _ _ 1 1 

\ \ t ; 


i i _ _ i 

i i i 

1 1 2 1 
f i i 


i 

i 

i 

i 

i 


. i i 

i i 

* n ! 

- 1 _ _ i 


! ptr loti 


t i i 

--> ! otr ! pt i > , • , 


i 

, ! pt r 


1 NIL ! 


» 

t 


t i i _ _ _ » 

i i • * 

i 
i 


i i _ ^ 

t i 

■ 


i i 


V 


V 


V 




a 1 1 r i but e value 


attribute value 


attribute value 


list of membe r 1 


list of member 2 


list 


of member 



(last member ) 



b) Structure of attribute value lists: 





_ i _ i 

! A 1 J 


i t _ i 

i i i 

: : A2i 

t _____ i _____ i 




• i 

! ! A(n) 

i _ _ t _____ 


1 Ptr 

i m i 








i i 

t _ „ i *■ r | 


i o t r -- 

_ i _ i 

• i 


* i i _____ i 

i i i "~ i 


" ^ , 


• otr i 

1 1 _ 1 

Iff 



i»i iii i i i 

i i i 

i i i 



V 



V 



V 



value st rue t ure 
of first attri but e 



value st rue ture 
of second attribute 



value s t rue ture 
of n t h attribute 



where At, A2,,,. represent the ATTRIBUTE IDs of correSDondiog 
values 



c ) Value s t rue ture: 



_____ ( _____ ( 

t iptr > 


_____!_ _ _ 1 

_____ f _____ , 

t ! Pt r-- > , , , 


t! NIL 


VALUE REPRE! 


SEMTATION, , ! 

■ 





where t represents the type of the value: 
t-0: single value 
t = 1 : range values 
t = 2: multiple values 

* is used as delimiter. 

Usage of M ***fl" for value representation indicates that value 
is not known. 
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ANNEX a 



Page 2 of 3 



WORKSHEET FOP OESIGN PHASE AND DESIGN REVIEW PHASE OF PROJECT » U 



STEP! PROBLEM AND PLANNED SOLUTION 

* I 



3 



a 



a) Define utilities for input 
of data base information: 
-FILL,EXTRACT,GETCHAR,SKIP 
-GET OLD FILE INFORMATION 
-STQRE,CLEARIN,CLEAR0UT 
-CARDIN, PRINTOUT 

-TYPIN, TYPOUT 
-FILE I 0, TERMINAL I 0 
-GET ATTRIBUTES AND VALUES 
-GET ATTRIBUTE, GET VALUE 
-FIND ATTRIBUTE 

b) other utilities: 

-SPACE, ERROR, DIAGNOSTICS 
-INITIALIZE all 

a) Define command capabilities 
-List all attributes (LISTA) 
-List all members (using only 
one attribute value oair 
chosen by user) (LISTALL) 

-List all members (printina 
all attribute value pairs of 
each member (LISTDBASE) 

-List all available commands 
(LISTC) 

-List all attribute value 
pairs of a sinole member 
(L I STM) 



ALTERNATE 

SOLUTIONS 



! MAN | 

DAY * HOURS ! EFPOR 
TIMEi/STEP! n 



COMMENT 



a/20 ! 

0900! a.O 



a/20 

1300 

a/21 

0900 



2.0 



J-Select a single member for ! 
{retrieval of particular values! 

!of interest (CONTROL) ! 

5-List a particular attribute ! 
lvalue pair of memoer in con- ! 
Strol (FA,ATTR) ! 

J-Find all oata base members ! 
{which with given conditions ! 

! (FIND, FINO ID) ! 

J-Switch output from terminal ! 

{to file and vice v e r s a ( SW I TCH ) ! 
!-Change key attribute (KEY) ! 
'-Obtain diagnostics (D) ! 

{-Super user mode (SU) ! 

!b) All subroutines which han- { 
{die commands are named accor- { 

! di ng 1 y , { 

Remarks: The last two functions are not 
effectively used for testing purposes, 
extensions of the program. 



;a/2i 

{1100 

i 

i 

required, however 
They also suoco r t 



they can be 
further 
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ANNEX A 



Page 3 of 3 



WORKSHEET FOR DESIGN PHASE AND DESIGN REVIE/v PHASE OF PROJECT n 4 



STEP 

n 



PROBLEM AND PLANNED SOLUTION 



alternate 

SOLUTIONS 



DAY 

TIM£ 



MAN | 

HOURS I ERROR 
/STEP! * 



COMMENT 



5 



Analysis of implementing FIND 
(FIND ID) function. 

Definiton of subroutines: 

-GET CONDITIONS 
-GET REGUEST 

-GET Q VALUE (functions which 
allow user to define Queries) 
-TEST CONDITIONS (evaluation 
o f auer i es ) 

Define functions to test for 
single discrete values: 

-r IND A I TR I BUTE 
-equal, matching 

-FIND M E M 6ER 

Define functions to orint re- 
sults: 

-*R I TE RESULTS 
-LlSTATTRIBUTE(value) 

All memoers w n i c h satisfy user 
defined conditions are first 
stored within a linked 1 i 3 1 • 
The result of all comoarisions 
are printed after all data 
members have been examined. 



4/22 

1230 



1 .5 



a/22 

1430 



6 



Design of retrieval functions 
for multiple values and range 
values: 

-COMPARE MANY 
-LLIMITrULIMIT 

(return upper and lower limit) 



a/23 

1530 



6 . 0 



4/25 
1 100 



7 {Design review and implementa- 
tion of improvements. 



4/27 

1300 

4/27 

2030 



1.5 
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Annex B 



Program Listing of Project U 4 
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MEM RE/»D INFORMATION FROM OLD INPUT FILE 
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ANNEX C 



Page 1 of 2 



WORKSHEET FOR COOING PHASE OF PROJECT n ; a 
Beginning of Coding (day/time) I 4/20/1400 
End of Coding (day/time) ! 5/02/1200 
Man hours : 24.5 (including punching of cards) 



1 ) 



COOING 

BEGIN ! END 

day/time i day/time 


PROGRAM PART 


ERROR 

a 


DAY 

TIME 


COMMENT 


04/20/1 «00 


04/20/1800 


Codina of primi- 
tives and utili- 
ties 






l) Record when error is 
detected. 

A great number of 
primitives could be 
copied from project *2. 


04/20/2030 


04/20/2330 


Punching caros 








04/21/1330 


04/21/1530 


Codina of co fl " 
mand func Cions 








04/21/1530 


04/21/1830 


Pune h i nq caras 


6 


1830 


C 9 


04/21/2030 


04/21/2130 


Coding of com- 

mand func Cions 
and punch i ng 
C a rds 








04/21/2230 


04/21/2330 


C od i ng of com- 
mand func t i ons 
ana punc h i nq 
cards 








04/22/1500 


04/22/1800 


Codina of 
ret r i eva 1 
func Cions 








04/22/1830 


04/22/1930 


Punching caros 








04/25/1330 


04/25/1630 


Coding of re* 
r e c r i e va 1 
func cions for 
multiple values 
and range values 
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annex c 



Page 2 of 2 



WORKSHEET FOR COOING PHASE OF PROJECT a : 4 
Beginning of Coding (day/time) : 4/20/1400 
End of Coding (day/ti^e) : 5/02/1200 
Man hours : 24.5 (including punching of cards) 



l ) 



COOING 

BEGIN ! ENO 

04Y/TIME | D4Y/TI,*£ 


PROGRAM PART { ERROR 'DAY 

: » : time 

• i 

• i 


COMMENT 


1 

I 


• 

i 




1) Record when error is 


1 

1 


i 

• 




detectea. 


04/25/21 00 ! 


Punchi na cares ! 






1 

1 


! 35 


2100 


A 1 


• 

1 


: 36 


2200 


A 1 


! 04/25/2200 
• 


1 

» 

• 






0*4/26/1530 ! 


1 

C od i ng o f J 






1 

1 


deougging aias ! 






1 

1 


and punching 






! 04/26/1630 

i 


cards ! 

• 






04/27/2030 ! 


• 

Changes accor- ! 






i 

» 


ding to oesign J 48 


2050 


C 1 7 


! 00/27/2100 
1 


review ! 

• 






05/02/ 1 100! 


• 

Coding of new ! 






• 

• 


Drocedures to I 




Change due to error 50. 


• 

• 


handle range ! 






• 

• 


value comDarison,’ 






i 

i 


and punching { 






{ 05/02/1200 

i 

« 


cards ! 

i 

i 
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ANNEX D 



Page 1 of tO 



WORKSHEET FOR DEBUGGING PHASE 
PROJECT ft : a DEBUG Run ft : 1 

Begin of Debug Run (day/time) : 04/20/2330 

End of Debug Run (day/time) : 04/21/2030 

ft of Debug Steps incl. in Debug Run: 4 CPU time for Debug run (sec): 0.0 

CPU time for necessary compiles (sec) : 19.39 

a) 3.93 b) 4.42 c) 5.34 o) 5.70 e) f) g) 

Man nours for this Debug Run : 2.5 (including preparation of debug run) 

MAN 1 ) 

STEP ! PROGRAM j OBJECTIVE AND EXPECTED 
ft J PART | RESULT 



ACTUAL 

RESULT 


DAY • HOURS 

tihe:/ste° 


ERROR 

ft 


comments 

AND CODED 
ERROR TYPES 










1 ) Record when 




4/20 






error 




2330 


1 .0 




occurs 


5 comp i 1 e 


4/21 








errors 


1030 




1 


Cb 








2 


Ai 








3 


A 1 




4/21 




a 


A 1 




1100 




5 


A 1 


1 c omo i 1 e 


4/21 








error 


1630 


0.5 








1900 




7 


C 1 7 




4/21 










1930 








1 compile 


4/21 








error 


1 q 30 


0.5 








2000 




8 


C12 




4/21 










2000 








O.K. 


4/21 










2000 


0.5 








4/21 










2030 









P r i mi- 
t i v e s 
and 

utili- 

ties 



P r i m i - 
t i ves / 
utili- 
ties^ 
i nput 
f unc t - 
ions 



Get error free comoile 



Get error free compile 



repeat step 2 



repeat step 2 
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ANNEX 0 



Page 2 of 10 



WORKSHEET FOR DEBUGGING PHASE 
PROJECT 0 : a DEBUG Run * : 2 

Begin of Debug Run (day/time) : 04/21/2130 

End of Debug Run (day/time) : 04/22/1830 

0 of Debug Steps i nc 1 • in Debug Run: 6 CPU time for Debug run (sec): 14.83 
CPU time for necessary compiles (sec) : 39.55 
a) 6.48 b) 5.52 c) 5.68 a) 4.47 e) 8.26 f) 9.14 g) 



Man hours for this Deoug Run : 5.5 (including preparation of decug run) 

MAN 1 ) 



STEP ! PROGRAM 
a ! ° AR T 

1 
1 


OBJECTIVE AND EXPECTED 

RESULT 


ACTUAL : DAY 
RESULT ITIME 
1 


1 HOURS 1 ERROR 

1 /STEP : 0 

1 


comments 

AND CODED 
ERROR TYPES 








1 

• 






1 ) Record wh« 


1 


A! 1 


Get error free compile 


2 comoileJ4/21 






error 




except 




er ror s ! 2 1 30 


0.5 




occurs 




ret rie- 




12130 




0 


C 7 




v a 1 




i 

i 




10 


A 1 




f unc t - 




1 4/21 










i ons 




12200 








2 




repeat steo 1 and check 


1 Drogram|4/21 












initial ization 


error 12200 


0.5 












12200 




1 1 


D 7 








1 4/21 














12230 








3 




repeat step 2 


5 program 14/21 














errors 12330 


2.5 












14/22 














11000 




12 


C 2 8 








11020 




13 


C21 








11030 




14 


C21 








1 1100 




15 


C28 








11110 




16 


D12 








1 4/22 














11200 








4 


A 1 1 


Get error free compile 


2 comp i 1 e 1 4/22 










except 


(including command 


e r ro r s 11200 


0.5 








retrie- 


f unc t i ons ) 


1 1200 




17 


C 6 




val 




11230 




18 


C 1 7 




f unc t - 




14/22 










i ons 




1 1230 








5 




repeat step 4 ana check 


2 program 1 4/22 












initial ization 


er ror s 11400 


1 .0 












11430 




19 


D 1 2 








14/22 














1 1500 




20 


C 2 7 


6 




repeat step 5 


O.K. 14/22 














1 1800 


0.5 
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'WORKSHEET FOR DEBUGGING PHASE 
PROJECT u : 4 DEBUG Run s : 3 

8egin of Debug Run (day/time) : 04/22/1830 

End of Debug Run (day/time) : 04/22/2200 

# of Debug Steps i nc 1 . in Debug Run: 2 CPU time for Debug run (sec): 11.93 

CPU time for necessary compiles (sec) : 17.75 
a) 8.71 b) 9.04 C ) d) e) f) g) 



Han hours for this Debug Run : 2.5 (including preparation of debug run) 

HA,n 1) 



STEP 


PROGRAM 


OBJECTIVE AND EXPECTED 


ACTUAL 


DAY 


! HOURS {ERROR J COMMENTS 




PART 


RESULT 


result 


TIME S /STEP ! 


: * 


: amo cooeo 












I 

1 


i 

i 


: ERROR TYPES 
















1) Record when 
















er ror 
















occurs 


1 


A 1 1 


Get error free comoile 


3 c oiro i 1 e 


4/22 










exceot 




e r ro r s 


1930 


0.5 




' 




f unc t - 






1945 




21 


A 1 




t i ons 






1950 




22 


A2 




to com- 






1 Q 55 




23 


C23 




pare 
range 
values 
or mu 1 - 
tide 
dis- 
crete 






4/22 










va 1 ues 






2000 








2 




repeat step 1 and examine 




4/22 












storaae of data base in- 
format ion 


O.K. 


2000 












(all attributes and values 
should be implemented 




4/22 












according to cesign) 




2200 
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WORKSHEET FOR DEBUGGING PHASE 
PROJECT 4 : a DEBUG Run n : a 

Begin of Debug Run (day/time) : 04/22/2200 

End of Debug Run (day/time) : 04/25/1100 

4 of Debug Steps incl. in Debug Run: 6 CPU tine for Debug run (sec): 43.47 
CPU time for necessary compiles (sec) : 73.14 
a) 11.36 b) 9.73 c) 9.84 d) 9.9e> e)l0.87 f)10.49 g) 10.89 



Man hours for this Debug Run : 7,5 (including oreoaration of debug run) 

MAN 1) 



STEP 

* 


PROGRAM 

PART 


OBJECTIVE AND EXPECTED 

RESULT 


ACTUAL 

RESULT 


DAY ! HOURS : EPROR 

time; /step: * 

i 

i 


COMMENTS 
AND CODED 
ERROR TYPES 
















1 ) Record when 


1 


A 1 1 


Check basic functions 


1 program 


4/22 






error 




exceot 


(exceot FIND function) 


error 


2200 


0.5 




occurs 




compa- 






2200 




24 


D 1 2 




rison 
















of mul- 
















tiple 
















values 
















o r 
















range 






4/22 










values 






2230 








2 




repeat steo 1 


l proaram 


4/22 














error 


2230 


3.0 














4/23 
















1230 




25 


D 3 


3 




repeat step 1 


1 program 


4/23 














error 


1230 


1.0 














1 300 




2b 


Di 3 










4/23 
















1330 








4 




repeat steo 1 


1 program 


4/23 














error 


1330 


1.0 














1400 




27 


D 1 5 










4/23 
















1430 








5 




repeat step 1 


2 program 


4/23 














e r ro r s 


1430 


1.0 














1500 




28 


D 1 2 










1500 




29 


C27 










4/23 
















1530 








6 




repeat steo 1 


O.K. 


4/24 
















1700 


1.0 














4/25 
















1 100 
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WORKSHEET FOR DEBUGGING PHASE 
PROJECT U : a DEBUG Run * : 5 

Begin of Debug Run (day/time) : 04/25/1630 

End of Debug Run (day/time) : 04/25/2400 

tf of Debug Steps incl. in Ddbug Run: 5 CPU time for Debug run (sec): 8*95 
CPU time for necessary compiles (sec) : 51*3 
a) 11.02 b) 11.48 c) 11.90 g) 9.37 e) 7.53 f) g) 



Han hours for this Debug Run : 5.5 (including preparation of debug run) 

MAN 1) 



STEP 

ts 


PROGRAM 

PART 


OBJECTIVE A NO EXPECTED 

RESULT 


ACTUAL 

RESULT 


DAY : HOURS ! error 

TIMEi/STEP! a 

• 


comments 

AND CODED 
ERROR TYPES 
















1 ) Record *ner 


1 


F i nd 


Check FIND function for 


3 program 


4/25 






error 




f unc t - 


trivial cases 


errors 


1 o 3 0 


1 .0 




occurs 




t i on 


Check input of conditions 




1630 




30 


A 1 




(single 


(0-4 conditions should oe 




1700 




31 


C27 




dis- 


stacked according to 




1730 




32 


G 1 2 




crete 


design) 




4/25 










values) 






1730 








2 




repeat step 1 


1 program 


4/25 












(using trace) 


error 


1700 


2.0 














2030 




33 


84 (Error 3 32 










4/25 






not correctea 










2030 






p rope r 1 y ) 


3 




repeat step 1 


1 program 


4/25 














error 


20 30 


0.5 














2100 




34 


C28 










4/25 
















2100 








4 




repeat step 1 


1 program 


4/25 














error 


2200 


0.5 














2230 




37 












4/25 
















2245 








5 




repeat step 1 


0 • K . 


4/25 
















2245 


1.5 














4/25 
















2400 









261 



ANNEX 0 
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WORKSHEET FOR DEBUGGING PHASE 
PROJECT n : 0 DEBUG Run * : 6 

Begin of Debug Run (day/time) : 00/25/2400 

End of Debug Run (day/time) : 00/26/1930 

* of Debug Steps incl. in Debug Run: 7 CPU time for Debug run (sec): 10.23 
CPU time for necessary compiles (sec) : 67.80 
a) 6.57 b) 12.88 c) 12.66 a) 6.66 e) 8.9b M10.27 g) 9.84 



Man hours for this Debuq Run : 0.0 (including preparation of debug run) 

*Ai\| 1) 



STEP 

* 


PROGRAM; OBJECTIVE AND EXPECTED 

PART ! " RESULT 

♦ 
i 


ACTUAL 

RESULT 


DAY 1 HOURS ! ERROR 
TIMEJ/STEP! * 

i 


COMMENTS 
AND CODED 
ERROR TYPES 




i 

i 




« 

« 




1 ) Record rfhen 


1 


All ! Get error free compile 


3 como i 1 e 


0/25! 




error 




! and test PINO function 


e r ro r s 


2000 i 2.0 




occurs 




! for trivial cases 




0/16! 








1 (all members listed must 




0900! 


30 


A 1 




! match conditions soeci- 




0910' 


39 


A 1 




! fied by input) 




0915! 


oo 


A 1 




• 

i 




o/26! 








• 

i 




1000 J 






2 


1 repeat step 1 


2 program 


0/26! 








* 

i 


errors 


1000! 2.0 








i 

i 




1050! 


01 


C29 




i 

i 




1100! 


02 


C20 




• 

i 




o/26! 








i 

♦ 




1200! 






3 


! repeat step 1 


1 program 


o/26! 1.0 








i 

i 


error 


1 330! 


03 


A 1 




» 

■ 




0/26! 








• 

i 




looo ! 






0 


J repeat step 1 


1 como i 1 e 


o/26! 








1 (including deougginq 


error 


1630! 1.0 








J aids) 




o/26 J 








« 

i 




1 730! 


40 


A 1 


5 


! repeat step o 


1 program 


o/26! 








■ 

■ 


error 


1730! 0.5 








i 

i 




1705! 


45 


A3 




■ 

• 




o/26! 








i 

i 




1800 ! 






6 


* repeat step 0 


1 proqram 


o/26! 








i 

• 


error 


1000! 0.5 








■ 

i 




1015! 


46 


A 1 




i 

i 




o/2b{ 








i 

i 




1830! 






7 


! repeat step o 


O.K. 


o/26! 








• 

i 




1030! 1.0 
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worksheet for DEBUGGING PHASE 
PROJECT * : a OEBUG Run a : 7 

Begin of Debug Run (day/time) : 04/26/2000 

End of Debug Run (day/time) : 04/26/2300 

tt of Debug Steps incl. in Debug Run: 2 CPU time for Debug run (sec): 7,0 
CPU time for necessary compiles (sec) : 18.14 

a) 9.73 b) 8.41 c ) d) e) f) g) 



Man hours for this Debug Run : 3.0 (including preparation of deoug run) 

MAN 1) 



STEP [PROGRAM j OBJECTIVE AND EXPECTED 

* [ PART [ RESULT 

1 < 

1 1 


! ACTUAL 
1 RESULT 


0AY [hOURSIERROR 

TIMEI/STEP! * 

1 

1 


comments 

AND COOED 
ERROR TYPES 


I 1 

1 t 




1 

1 




1 ) Record when 


1 1 

« 1 




1 

1 




error 


l 1 

l 1 




1 

1 




occurs 


1 ) All ! Test FIND function for 


! 1 p r og r am 


4/26 J 






5 ! various inout (including 


1 e r ror 


2000; 2.0 






! J also extreme conditions) 




21201 


47 


C28 


1 (all queries should be 




• 

i 






! J answered accordino 




4 / 2 6 ; 






S ! design) 

• i 




2200; 

• 






• i 

2 J 5 repeat steo 1 


! O.K. 


4/26; 






i i 

i • 




2200; 1.0 






• • 

i i 




4/26; 






• i 

• i 

i i 

i < 




2 30 0 1 

i 

i 
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WORKSHEET FOR DEBUGGING PHASE 
PROJECT U : a DEBUG Run z : 8 

Begin of Debug Run (day/time) 2 04/27/1500 

End of Debug Run (day/time) : 04/27/1900 

U of Debug Steps i nc 1 • in Debug Run: 1 CPU time for Debug run (sec): 11,98 

CPU time for necessary compiles (sec) : 10.02 

a) 10.02 b) c) a) e) f) g) 



Man hours for this Debuo Run : 3.0 (including preparation of debua run) 

MAN 1 ) 



STEPJPROGRA* 
* J PART 

» 

i 


08JECTIVE 4 NO EXPECTED 

RESULT 


ACTUAL 

RESULT 


DAY J HOURS ; ERROR 
TIME! /STEP! « 

i 

• 


COMMENTS 
AND CODED 
ERROR TYPES 


i 

i 

i 

i 

i 

i : aii 
< 
i 
i 
• 

• 

i 


Repeat Debua Run *7 under 
CP/CMS 

(same results expected) 


O.K . 


a/27 

1S00 

a/27 

1900 


• 

1 

• 

1 

1 

I 

1 

3.0 ! 
1 
1 
1 
1 


l ) Record when 
error 
occurs 
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WORKShEET FOR DEBUGGING PHASE 



PROJECT n : a DEBUG Run * : 9 

Begin of Debug Run (day/time) : 04/29/1400 

End of Debug Run (day/time) : 04/29/1600 

tt of Debug Steps incl. in Debug Run: 1 CPU tine for Debug run (sec): 19.58 

CPU tine for necessary compiles (sec) : 9,4o 
a ) 9.46 b) c) c) e) f) g) 



Man hours for this Debug p un 

stepiprogram; objective AND 

# ! PART J RESULT 



2.0 (including 



d reoa r at ion of 
VAN 



debug 
1 ) 



EXPECTED 



ACTUAL 

RESULT 



CAY J HOURS J ERROR 
TIME! /STEP} 3 



run ) 



COMMENTS 
AND COOED 
ERROR TYPES 



A 1 1 


i 

i 

t 

i 

i 

i 

i 

i 


Various tests under 


O.K. 


4/28 




• 

i 


CP/CMS: 




1400 




i 

i 


-Test initialization 








i 

i 


-Test LISTA (list of all 








i 

i 


attributes ) 








i 

• 


-Test LISTC (list all 








i 

i 


available commands) 








i 

i 


-Test LISTM (check for 








i 

• 


all members and illegal 








• 

• 


i npu t ) 








« 

i 


-Check program for input 








i 

i 


of undefined commands 








i 

t 


-Test CONTROL (check for 








i 

• 


i 1 1 ega 1 i nout also) 








i 

i 


-Test functions FA and 








i 

i 


ATTR for several members 








t 

i 


in con t ro 1 








t 

• 


-Test SWITCH (examine out- 








i 

i 


put file) 








« 

i 


-Test KEY (change of key 








i 

i 


attribute as desirea by 








i 

i 


user ) 








i 

i 


-Test LIST DBASE (al 1 mem- 








i 

i 


bers are listed with all 








i 

i 


all at t ri but e va 1 ue 








i 

i 


pa i r s ) 








i 

i 


-Test LISTALL (memoers are 








i 

i 


listed only by key attri- 








i 

i 


bu t e ) 








i 

i 


-Test FIND function (using 








i 

t 


various input combi- 




4/29 




• 

i 


nations) 




1 o 0 0 



1) Record when 
error 
occurs 
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WORKSHEET FOR DEBUGGING PHASE 
PROJECT 0 : a DEBUG Run * : 10 

Begin of Debug Run (day/time) : 04/28/1600 

End of Debug Run (aay/time) : 04/28/1800 

0 of Debug Steos incl. in Debug Run: 1 CPU time for Debug run (sec): 12,08 

CPU time for necessary comoiles (sec) : P.46 
a)P.46 b) c) a) e) f) g) 



>4 an hours for this Debug Run : 2.0 (including preparation of debua run) 

.HAN 1) 



STEP 

n 


PROGRAM 

PART 


OBJECTIVE AND EXPECTED 

RESULT 


ACTUAL 

RESULT 


DAY ' HOURS ! ERROR 

TIMEJ/STEP! 0 
1 
1 


COMMENTS 
AND CODED 
ERROR TYPES 


1 


A 1 1 


Various test s under 
CP/CMS: 

a) Test SU (super user 
f u n c t ion) 


1 program 
error 


4/28 

1600 

1620 


2.0 


UR 


1 ) Record when 
error 
occurs 

C 27 






b) Test FIND function 
for 0 cond i t i ons (all 
members should oe listed) 


O.K. 














c) lest FIND amo FIND ID 
using 1-4 different con- 
ditions (check correct- 
ness of program especially 
multiple values and range 
values) 


O.K . 










/ 




d) Test KEY function for 
illegal inputs (proaram 
should provide an appro- 
priate error message) 


O.K, 


4/28 
18 50 
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ANNEX El 



Page 1 



of 2 



ERROR LISTING 



PROJECT * : 4 



TIME 

scent to 
solve the 
ERROR 
( M an 
min.) 



1 


Debuggi ng 


Coding 


C 6 


5 


2 


Debugg i ng 


Cod i ng 


A 1 


5 


3 


Debugq i ng 


Cod i ng 


A 1 


5 


4 


Oebugg i ng 


Coding 


A 1 


5 


S 


Debugg i ng 


Coding 


A 1 


5 


6 


Coding 


Cod i nq 


C 9 


S 


7 


Debuggi ng 


Coding 


Cl 7 


5 


3 


Deouqg i ng 


Coding 


C 1 2 


5 


9 


Debuggi ng 


Coding 


C 7 


5 


10 


Debugg i ng 


Coding 


A 1 


5 


1 1 


Debuggi ng 


Design 


D 7 


S 


12 


Debuggi ng 


Coding 


C28 


S 


13 


Debuggi ng 


Cod i ng 


C21 


15 


14 


Debugq i ng 


Cod i ng 


C21 


15 


IS 


Debugq i nq 


Coding 


C23 


10 


16 


Debugq i ng 


Design 


D 1 2 


25 


17 


Debugq i ng 


Coding 


Co 


5 


18 


Debuggi ng 


Coding 


C 1 7 


5 


1° 


Debugg i ng 


Design 


D 1 2 


30 


20 


Debugg i ng 


Coding 


C27 


1 0 


21 


Debuggi ng 


Coding 


A 1 


5 


22 


Debuggi ng 


Cod i ng 


A 2 


5 


23 


Debugg i ng 


Cod i ng 


C23 


5 


24 


Debugg i ng 


Design 


D 1 2 


10 


25 


Debugg i ng 


Design 


D 3 


15 


26 


Debugg i ng 


Design 


D 1 3 


30 


27 


Debugg i ng 


Design 


D 1 5 


20 


28 


Debuggi ng 


Design 


D 1 2 


30 


29 


Debugg i ng 


Codi ng 


C27 


5 


30 


Debugg i ng 


Coding 


A 1 


5 



Begin of Project (day/time) : 04/19/1400 
End of Project (day/time) : 04/03/1800 
Man hours for total project : 101.0 



ERROR 


PHASE 


PHASE 


ERROR 


u 


in which 


in which 


TYPE 




ERROR was 


ERROR was 


(see ANNEX F) 




dis- 


made 






covered 







n of OTHER 
STATEMENTS OR 
PARTS Or THE 
PROGRAM 
AFFECTED 
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ANNEX El 



Page 2 



o f 2 



ERROR LISTING 



PROJECT 0 : a 

Begin of Project (day/time) 
End of Project (day/time) 
Man hours for total project 



04/19/ 1400 
04/03/1800 
101,0 



ERROR 

u 


PHASE 
in which 
ERROR was 
dis- 
covered 


PHASE 

in which 

ERROR was 
made 


£RR0R 

TYPE 

(see ANNEX F) 


TIME 

scent to 
solve the 
ERROR 
( M an 
min.) 


31 


Oeouaa i ng 


C o d i n g 


C27 


1 0 


32 


Debuggi ng 


Design 


012 


90 


33 


Deouaq i ng 


Debuga ing 


84 


15 


34 


Debuga i ng 


Coding 


C28 


15 


35 


Coding 


Coding 


A 1 


5 


36 


Cod i ng 


Cod i ng 


A 1 


5 


37 


Debugg i ng 


Deouga i ng 


A 1 


5 


38 


Debuggi ng 


Coding 


A 1 


5 


39 


Debuggi nq 


Coding 


A 1 


5 


40 


Debugg i ng 


Coding 


A 1 


5 


41 


Debuga i ng 


Cod i ng 


C28 


5 


42 


Debugqing 


C o d i n g 


C28 


5 


43 


Debugg i nq 


Coding 


A 1 


5 


44 


Debuggi ng 


Coding 


A 1 


5 


45 


Debugg i na 


Debugg i ng 


A3 


5 


46 


Debugg i ng 


Coding 


A 1 


10 


47 


Debugg i nq 


Coding 


C28 


10 


48 


Coding 


Coding 


C 1 7 


5 


4R 


Debugg i ng 


Coding 


C27 


30 


50 


Testing 


Design 


D3 


120 



tt of OTHER 
STATEMENTS OR 
PARTS OF THE 
PROGRAM 
AFFECTED 
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ANNEX E2 
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ERROR LISTING (COMMENTS) 



ERROR 

tt 


DAY 

time 


1 


04/21 

1030 


2 


1030 


3 


1030 


4 


1030 


s 


1030 


b 


1830 


7 


1900 


8 


2000 


9 


2130 


10 


2130 


1 1 


2200 


12 


04/22 

1000 


13 


1020 


14 


1030 


15 


1 100 


16 


1110 


17 


1200 


18 


1230 


1 Q 


1430 


20 


1500 


21 


1945 


22 


1950 


23 


1955 


24 


2200 


25 


04/23 

1230 


26 


1300 


27 


1400 


28 


1500 


2R 


1500 


30 


04/25 

1630 



COMMENTS 

(EVIDENCE, THOUGHTS, WHY WAS THE ERROR MADE? 
WHY AND HOW WAS THE ERROR DISCOVERED? 

ERROR BLOCKING, etc.) 



Errors #1 thru 5 are due to influence of fatigue. 



Found while checking previously written code during 
ounching of cards. 

Different variaole name assumed. 

Missing mandatory declaration. 



Errors *14 thru lb are due to insufficient des< checking. 



Upper case <ey pressed while ounching caras. 
Usage of "THEN 1 * instead of "DQ H . (wrong format) 
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ANNEX £2 



Page 2 of 2 



ERROR LISTING (COMMENTS) 



ERROR 

n 


DAY 

TIME 


COMMENTS 

(EVIOENCE, THOUGHTS, WHY WAS THE ERROR MADE? 
WHY AND HOW WAS THE ERROR DISCOVERED? 

ERROR BLOCKING, etc.) 


31 


04/25 

1700 




32 


1730 


No proper desk checking. (Error could have been 


33 


2030 


avoided by des* checking.) 

Inapo roor i a t e correction of error «32. Debugging 


3a 


2100 


results were not interpreted correctly. 
Forgotten " ♦ 1 “ . 


35 


2100 


Errors s35 and 36 are due to f a t i gue . ( t r i v i a 1 errors) 


36 


2200 




37 


2230 




38 


04/26 

0900 


Errors « 3 8 thru 40 are due to lack of concentration 


39 


0910 


while Punching cards. 


40 


0915 




41 


1050 




42 


1 100 




43 


1330 




44 


1730 




45 


1745 




46 


1815 




47 


2120 


Double negation in connection with logical AND 


48 


04/27 

2050 


could have oeen tested bv usage of a truth taole. 
The error could have been avoided this way during 
a desx test. 

Found while punching cards. 


49 


04/28 

1620 




50 


05/01 

1700 


It was surprising to the experiment Programmer 






that using the EBCDIC character set the 
following comparison will be evaluated "TRUE’*: 

n — m y m ^ n 

Error * 50 was made Decause this relationsnio 
was not known. 
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ANNEX F 



FINAL STATISTICS 



PROJECT * 4 



FINAL STATISTICS 

Project name : DATA RETRIEVAL SYSTEM 
Short description: 

The program is designed for usage under CP/CMS* It 
expects an input file labeled ''DBASE INPUT'* to contain data 
base information in a particular format such that the pro- 
gram can read all information and store it into memory. 
After all data base members are defined by input ana imple- 
mented within linked lists the user mav operate uoon the 
data case using functions from a previously defined set of 
f unc t i ons • 

The program has been designed and implemented in such a 
manner that it will oe easy to implement more functions or 
to extent the currently defined limits of the program. 

Quantitative measures: 



1. * of source statements : 1084 

2. Total man hours for project : 101.0 

3. M an hours spent in 

a ) Design : 24 . 0 

b) Coding : 24.5 

c ) Deouggi ng : 41.5 

d) Testing : 11.0 

4. CPU time for compiles: 343.57 sec. 

5. CPU time for debug runs: 140.05 sec. 

6. CPU time for test runs: 33.04 sec. 

7. a of test and deoug runs: 13 

8. a of test and debug steps: 43 
P. 4 of errors found: 50 

10, Total man hours used to correct errors: 11.25 
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ANNEX F 



FINAL STATISTICS 



11. Er ror Detection: 

Mean time between error detections: 87.7 man min. 



of 

e r ror s 
t 



35 



30 



25 



20 



15 



10 



1 



★ * * * 

* * * * 

* * ★ * * 

* * * * * * ★ 

* * * * * * * * * * *** *** *** man hours 



1 



1 1 



12 13 



23456789 10 

TIME between error detections 

(measured from point in time of detection of previous error) 



project t 



i me 
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ANNEX F 



FINAL STATISTICS 



12* Error Correction: 

Mean time to correct an error: 13.3 man min. 



A of 
Errors 
co r rec t ed . 
T 

; * 

3 0 + * 



25 



20 



15 



10 



5 



1 ! 

-t 

1 




TIME TO CORRECT ERRORS 
(measured from point in time of detection) 



* 

120 180 

man minutes 
of project time 
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ANNEX F 



FINAL STATISTICS 



1 3 • 


When 


errors 


were 


f ouno : 










a ) 


u 


o f 


errors 


found 


du r i ng 


design chase: 


0 




0,0 


b) 


n 


of 


e r ro ps 


found 


during 


design review: 


0 




0.0 


c) 


u 


0 f 


errors 


found 


during 


coding: 


4 




8.0 


d) 


n 


0 f 


e r ror s 


found 


during 


debuggi ng : 


45 


- 


90.0 


e) 


n 


0 f 


e r ro r s 


found 


during 


writing of 


















test procedures: 


0 




0.0 
















50 






f ) 


z 


o f 


e r ro rs 


found 


du r i ng 


testing: 


1 


= 


2.0 



14. 


//hen 


er rors 


were 


made: 










a ) 


9 


o f 


e r ro r s 


made 


during 


design chase: 


10 


- 


20.0 


o ) 


9 


0 f 


e r ro r s 


made 


during 


design review: 


0 


= 


0.0 


c) 


a 


o f 


e r ro r s 


made 


dur i ng 


coding: 


37 


s 


7 4.0 


d ) 


9 


o f 


e r ror s 


made 


durina 


deouag i na : 


3 


- 


b.O 


e) 


a 


o f 


e r ro r s 


made 


du r i ng 


writing of 


















test procedures : 


0 


s 


0.0 


f ) 


9 


o f 


e r ro r s 


made 


durina 


test i ng : 


0 


= 


0.0 



50 
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ANNEX F 



FINAL STATISTICS 



IS. TIME HISTORY GRAPHS : 



n of 
Errors 
T 

Error T ypes : 

A: Clerical error 
0: Debugging error 
C : Cod i ng error 
0: Design error 

A 

CO A AA 

A 0C C C AC A 
AO 00 AO0AACAA C CC 0 

2 20 uo 60 ao 100 

NUMBER OF ERRORS FOUNO VS PROJECT TIME 



: a c 

J A OCC 

: A CACCC 

1! C CCCCOO 



A of 
E r r o r s 

T 

10 * 



5 + 
• 

i 

i 

i ; 



2 



20 



* 

* * * * 

* * * * 
****** 



<40 



* 

* 

* * 

* - 



* 

* * 
* * 

- ♦ 



* * * 

***** * 

******** * 

60 80 



+ - ♦ 
1 00 



NUMBER of ERRORS CORPECTEO VS PROJECT TIME 



man 

hours 



man 

hours 
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ANNEX F 



ERROR CATEGORIES AND TYPES 



1. Design Errors 



The follwing types of errors apply to both categories 
"System Design Errors" and "Program Design Errors": 

D1 : Communi cat i on Error 

02 : Design Negligence 

03 : Forgotten Cases or Steps 

0^4 : Timing Problems 

D5 : Errors in I/O Concepts 

06 : Data Design Error 

07 : Initialization Error 

D0 : Inadequate Checking 

D 9 : Extreme Conditions Neglected 

DlO: Seauencing Error 

Dll: Indexing Error 

D12: Loop Control Errors 

D13: Misuse of Boolean Expression 

D14: Mathematical Error 

D 1 S : Reo r esen t a t i on Error 

D 1 6 : Misunderstanding of Proolem Specifications 

D 1 7 : Other Design Errors 



2 . Coding E r ro r s 



Cl 
C2 
C 3 
CU 
C5 
Co 
C 7 
C8 
C 9 
CIO 
Cl 1 
C 1 2 
C 1 3 

C14 
C 1 5 
C l 6 
C 1 7 
C18 
C l 9 
C 20 
C21 
C22 
C23 
C24 
C25 
C26 



Misunderstanding of Design 

Meg 1 i gence 

I/O Fo rmat Error 

Misplaced Data Declaration 

Multiple Data Declarations 

Missing Data Declaration 

Inadequate Data 

Initialization Error 

Error in Parameter Passing 

Inadequate or Forgotten Checking 

Levs 1 Prob 1 ems 

Missing Declarations of 31ock Limits 

Case selection error 

GO TO Prob 1 ems 

Comment Error 

Forgotten Delimiter 

Inconsistency in Naming 

rtrong Use of Nested IF Statements 

Indexing Error 

Inconsistent Use of Variables or Data 

Seauencing Error 

Flag Usage Problems 

Syntax Error 

Loop Control Error 

Incorrect Exit from Subroutines 

Language Usace Proolems 
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ANNEX F 



ERROR CATEGORIES AND TYPES 



C27: Forgotten Statements 

C28: Representation Error 

C£R: Control Sequence Error 

C30: Incorrect Subroutine Usage 

C3I: Other Coding Errors 



3. Clerical Errors 

A 1 : M anual Error 

A£ : Men t a 1 Error 

A3 : Procedural Errors 

A 4 : Other Clerical Errors 



4. Debugging Errors 



B1 : Inaooroori ate Use of Debugging Tools 

BE : Insufficient or Inaooroon ate Selection 

of Test Cases or Test Data 
B3 : Misinterpretation of Deougging Results 

B 4 : Mi sinteroretat ion of Error Source 

85 : Neg 1 i qence 

Bo : Other Deouqgina Errors 



5. Testing Errors 



Tl : Inadeauate Test Case(s) or Test Data 

T2 : Misinterpretation of Test Results 

T3 2 Mi sinteroretat ion of Program Specification 

T 4 : Neg 1 i gene e 

T5 2 Other Testing Errors 
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ANNEX G 



Rage l o* 34 



OIRECTEO GRAPH REPRESENTATION 

PROJECT * : 4 

Program Da rt : CLEARIN, CLEAPOUT, CARDIN, PRINTOUT, 



TYPIN, TYPOUT, PILE I/O, TERMINAL I/O, 

GET OLD FILE INFORMATION, *RITE N El« FILE 
INFORMATION, CAPl, CAR2, COR 1 * CDR2, 
SETCARl, 3ETCAR2, SETCDP1, SETCDR2, FREE 1 , 
FREE2, PINO ID, GETOVALUE, *AIN 



NUMBER CF NCC6S : 
NUMBER CF ARCS : 
NUMBER CF PATHS: 



COMPLEXI TY MEASURES: 



NUMBER OF STATEMENTS 



CYCLCMATIC NUMBER: 



2 

1 

l 



V ( G ) = 1 





SUM 



2 . CCOOOO 



reachability incex 

CF CIRECTEC GRAPH: 



i.cccooc 



NOTE: All 23 subroutines ‘nave the same structure 
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ANNEX G 



Pace 2 o f ^4 



DIRECTED GRAPH R£PR£ SEN T A T I ON 

PROJECT 9 : 4 

Program oart : ALLOCATE! * ^LLOCATEE/ CONTROL 



COMPLEXITY MEASURES: 



NUMBER OF STATEMENTS: 8 - 



NUMBER CF NCCcSs 
NUMBER CF ARCS : 
NUMBER CF PAThS: 
CYCLCMATIC NUMBER 



A 

3 

2 

V ( C* ) * 



Q 



1 



REACHABILITY CF NCCcS: 
NCCE 1 : 1 

NCCE 2 : 1 

NCCE 3 • 1 

NCCE A : 

SUM: A.CCGCCO 

R E AC HA 6 ILITY INDEX 
CF CIRcCTEC GRAPH : 



l.CCCOOO 
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ANNEX G 



Page 3 of 34 



DIRECTED GRAPH REPRESENTATION 

PROJECT a : a 

Program oart : SPACE 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: a 

NUMBS R CF NCC E S : 4 

NUMBER CF ARCS : 4 

NUMBER CF PATHS: 2 

CY Cl CM AT IC NUMBER : V ( G ) = 



2 



REACHABILITY CF NCCES: 



NOC5 
NC Cc 
NCCE 
NC CE 



SUM : 6.CC3000 

REACHABILITY INDEX 
CF DIRECTED GRAPH: 1.SCCOC3 



o 
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ANNEX G 



p ane '4 of 3 4 



DIRECTED GRAPH REPRESENTATION 

PROJECT a : a 

Program oart : INITIALIZE!, INITIALIZES 



COMPLEXITY MEASURES: 



NUMBER OF STATEMENTS: 7 - p 

NUHBER CF NCCES: 5 

NUMBER CF ARCS : 5 

Mj > 6 E R CF FATI-S: 2 

CYCLCMATIC NUMBER: V(G) = 



2 



REACHABILITY 
NClE I 

NCCE 2 

NC Cc 2 

NCCE R 

NCCE 5 



CF NCDES: 

1 

2 

I 



SUM: 6.CCCCCG 

REACHA8 IL ITY I N C E X 
CF 0IR5CTEC GRA P n : i. 399999 
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annex G 



Page 5 of 34 



DIRECTED GRAPm REPRESENTATION 

PROJECT « : 4 

Program Dart : INITIALIZE ALL 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: 56 
NUMBER CF NCCES: 9 

NUMBER CF ARCS : II 

NUMBER CF PATHS: 8 

CYCLCMATIC NUMBER: V(G) = 4 



REACHABILITY CF NCCES: 



NGCE 
NC CE 
NGCE 
NGCE 
NC CE 
NOCS 
NC CE 
NGCE 
NCCE 



SUM : 35.CCCCC 

reachability incex 

Or OIRECTEC GRAPH: 



422221 
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ANNEX G 



? aqe 3 c f 34 



DIRECTED GRAPH pERRESEN T A T I ON 

PROJECT = : 4 

Program Dart : ERROR 



COMPLEXITY MEASURES: 

NUM0EW OF STATEMENTS: 54 
NUMBER CF NCCES: 6 

NUMBER CF ARCS : 5 

NUMBER CF PATHS: A 

CYClCMATIC NUMBER : V(G) = 



R E AC HA 6 I L I TY CF NCCES: 
NCCc I : 1 

NCCc 2 : I 

NCCc 3 : I 

NC CE A : 1 

NCCE 5 : I 

tNC CE 6 : 1 




SUM: 6*CCCCCC 

FEACHAS IL ITY INCEX 
CF CIRECTEC GRAPH : L.CCCOOO 
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ANNEX G 



Paae 7 of 34 



DIRECTED GRAPH REPRESENTATION 

PROJECT a : a 

Proaram cart : DIAGNOSE 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: 83 

NUMBER CF NCCES: 16 

NUH6ER CF ARCS : 23 

NUH6EB CF PATHS: 40 

CYCLCMATIC NUM8ER: V(G) = 



REACHABILITY CF NCCES: 



NCCE 
NCCE 
NGCE 
NO Cc 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 



1 

2 

*3 

4 

5 

6 
7 

a 

s 

LC 

11 

12 

13 

14 

15 
lc 



1 

1 

c 

2 

1 

4 

2 

1C 

c 

1C 
2 C 
10 
2 C 
2C 
20 
2C 



SU^: 151.0030 

REACHAe IL ITY INCEX 
CF DIRECTEC GRAPH: 



*• Error 9 12 



9.427503 
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ANNEX G 



Paq? 6 of 3 4 



OIRECTE0 GRAPH REPRESENTATION 

PROJECT a : 4 

Program oart : EXTRACT, F I L L <- L I $ T A , L I 5 T C , 

FIND VALUE LENGTH 



complexity measures: 

NUMREP OF STATEMENTS: 6-13 

NUMBER CF NCCES: 5 

NUMBER CF ARCS : 5 

NUN6ER CF PATHS: 2 

CYCLCVATIC NUMBER : V ( G ) * 2 



REACHABILITY 
NCCE 1 

NCCE 2 

NC C t 2 

NCCE 4 

NGCE 5 



CF 



NCCcS: 

1 

2 
1 
2 
2 



SUN: 3.CCCC00 

REACHAB IL ITY INCEX 
CF DIRECTED GRAPH: 1.599999 




LISTALLf 
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ANNEX G 



Paae R of 34 



DIRECTED GRAPh REPRESENT A T [ON 

PROJECT * : a 

Program oart : GFTChar 



COMPLEXITY MEASURES: 
NUMBER OF STATEMENTS: 21 

NUMBER CF NCCES: 12 

NUMBER CF ARCS : 15 

NUNbER CF PATI-5: 16 

CYCLCMATIC NUMBER: V(G)= 



RcAChAQ ILITY CF NCCES: 
1 
1 
1 
2 
2 
14 
12 
6 

12 
12 
U 
ic 



95 * CO C 00 
R E AC H A E IL IT V INCH* 

CF CIR6CTEC GRAPE: 7 



NCCE 


1 


NCCE 


2 


NCCE 


a 


NCCE 


4 


NOCE 


5 


NCCE 


t 


NCCE 


7 


NCCE 


s 


NCCE 


c 


NCCc 


10 


NCCE 


11 


NG CE 


12 



SUP : 
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AfJf'jfc < G 



Page 10 of 34 



DIRECTED GRAPH °£PRES£NT A T ION 

PROJECT s : 4 

Program oart : SUPER USER 



COMPLEXITY MEASURES: 



NUMBER OF STATEMENTS: 4R 



NUMBER CF NCCES: 
NUMBER CF ARCS : 
NUNBcP CF PATHS: 
CYCLCMATIC NUMBER 



14 

17 

12 

V(G> = 



5 



PE AC HA 3 IlITY 
NCCE 1 : 

NCCc 2 : 

\C CE 3 : 

iNCCE 4 

NCCE 5 : 

NCCE 6 : 

NCCE 7 : 

NC CE 6 : 

NCCE 9 : 

NC C E 1 C : 



CF NCC 
1 
1 
1 
2 
2 



ES: 



4 

4 

4 

6 



NC CE 
NC CE 
NC l E 
NCC E 



1 1 
12 
L 2 
14 



4 



12 

12 



SUM : 6 5 . C 3 COO 

SEACHA 3 ILITY INCEX 
CF D IP EC T E C GRAPH : 4.642857 
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annex r» 



Page 11 o f i ^ 



DIRECTED GRAPH REPRESENTATION 



PROJECT ^ : j 



Program cart : STORE 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: 1 $ 

NUMBER CF NCCES: 7 

NUMBER CF ARCS : 3 

NUNeER C-F PATHS: 7 

CYCLCMATIC NUMBER: V ( C- ) = 



REACHABILITY CF 



NCCE 

NCCE 

\CCE 

NCCE 

NCCE 

NCCE 

NCCE 



NCCES: 

7 

£ 

i 

£ 

£ 

7 



SUM : 33 .CC 000 

REACi-Ac IL I TY INC 6 X 
CF 0 IRECTEC GRAPH : 
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ANNEX G 



P aae 12 o f 34 



DIRECTED GPAPm P E P R E S E N 7 a T I 0 N 

PROJECT a : a 

ProGram oart I GET ATTRIBUTES A NO VALUES 



a) « of nodes: ° 

b) a 0 f arcs : 11 

c) 3 of statements: ^ 0 

a) 3 of c a t R s : * 

e) R e a c h a o i I i t v ! * 

f) Cyclo^atic nurrer: 

* N u t c e r of o a r n s 

ana reacnaoi 1 i tv 
are very large. 
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ANNEX G 



3 ao9 13 of 3-i 



DIRECTED GR a p w REPOSE* T A T I ON 

PROJECT a : a 

Program cart : GET ATTRIBUTE 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: 23 

NUNBcP CF NCCES: 15 

NUB9EP CF ARCS s 10 

NUN0EF CF PATHS: 16 

CYCICMATIC NUN8EP: V(G)= 

REACHABILITY CF NCCE3: 



NCCE 

NCCE 

NCCE 

NCCE 

NGCE 

NCCE 

NGCE 

NCCE 

NGCE 

NCCE 

NCCE 

NGCE 

NCCE 

NCCE 

NCCE 



1 



4 

5 

6 
7 
6 
c 

1C 
11 
1 2 

13 

14 

15 



1 

1 

1 

1 

2 

2 

4 

A 

4 

2 

6 

6 

6 

12 

12 



SLY: 64.00000 

REACHAS I L IT Y INCcX 
CF DIPECTEC GRAPH: 



4.26c666 
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ANNEX G 



Paa® l 4 or 34 



DIRECTED GRAPH REPRESENTATION 

PROJECT 3 : a 

Program cart : GET VALUE 



a ) * o f nodes : l 7 

b) * of arcs : 23 

c) - of statements: 37 

o ) a of catns: * 

e) Peac n ac H i t v : * 

f ) CvC 5 O'nat i C °uncer: o 

* Numoer of o a t b s 
ana r® ac o ao i 1 i tv 
are very larae. 
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ANNEX G 



F aae 15 o f 3 a 



OIRECTEO GRAPH u E°RESENTATION 



PROJECT £ : a 

Program oart : FI .'JO ATTRIBUTE 



COMPLEXITY MEASURES: 



NUMBER OF STATEMENTS: IS 



NUMBER CP NCCES: 
NUMBER CF ARCS s 
NUMBER CF PATHS: 
CYCICMATIC NUMBER 



6 

7 

4 

V ( G ) » 



3 



REACHABILITY CF 
NCCE l : 

NCCE 2 : 

NGCE 3 : 

NCCE 4 : 

NCCE 3 : 

NGCE 6 : 



NOCSS: 

1 

2 
1 



c 

2 



A 



SUM : 12. CO 000 

REACHAE il ity incsx 
CF CIRECTEC CPAPH: 2.CCC000 




292 




ANNEX G 



Paae lo ot 3 J 



DIRECTED bRAP m REPRESENTATION 

PROJECT 3 : a 

Program cart : SKIP, FIND CO^AnD, GET CONDITIONS, 
TEST ALL 



COMPLEX IT/ MEASURES: 



NUMBER OF STATEMENTS: o - 

NUMBER CF iN C C 8 5 : 4 

NUMBER CF ARCS : 4 

NUMBER CF PATHS: 2 

CVCLC VATIC iN CHEER : V(G) = 



12 



2 



3EACFA8 IL ITV 
ISGCE I : 

NCCE 2 : 

\CCE 3 : 

NCCE 4 : 



CF NCCSS: 
1 
2 
X 



SGV: 6.CCCCC0 

R 6 AC HA 6 IL ITV INCSX 
CF 0 IRcCTSC GRAPH : 



l.SCCOCJ 
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ANNEX G 



Pace 17 of 34 



DIRECTED GRAPH °E p RESEN T A TI ON 

PROJECT s : a 

Program oart : INTERPRET 



COMPLEXITY ^EASUkES : 



NUMBER OF STATEMENTS: 

NU/* 6 ER CF NCCES: 13 

nuybep cf arcs : 17 

-NUMBER CF PATHS: 16 

CYCLC^ATIC MNBEP: 

V ( G ) = 1 

REACHABILITY CF NCCES 



NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 



1 

2 
3 
A 

6 

7 

8 
9 

1C 
1 1 
12 

13 

14 

15 

16 

17 

18 




SLM : 18 .CCCCC 

REACHABILITY INCEX 
CF OIRECTEC GRAPH: l.CCCOOO 
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ANNEX G 



Pace 18 of 34 



DIRECTED GF APh pe^PESEnT a T ION 

PROJECT 3 : 4 

Program oar* : SHIFT 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: Cl 
NUMBER CF NCC8S: 11 

NUMBER CF A SC S : 13 

NUN6EP CF PATHS: 8 

CYClCYATIC NUM5EP: V(G1= 4 



SS AC HABILI7Y CF NCOES: 



NCCE 
NCCE 
NCCE 
NO Cc 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 
NCCE 



6 

7 

8 
9 

1C 

11 



SUM: A2.CCCCC 

^EaCHA3 II ITY INCH* 

CF DIPECTEC CF APH : 



3.518181 
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ANNEX G 



Pacp 1° o t 3 A 



DIRECTED GRAPH P£PPF. SEN T A T I ON 

PROJECT 2 : U 

Program oart : L15TDBA$E 



COMPLEXITY MEASURES: 



NUMBER OF S f A I E w E N T S : l t 

NUMBER CF NCCES: 6 

NUMBER CF ARCS : 7 

NUMBER CF PATHS: A 

CYCLCMAT IC NUMBER : V ( G ) = 



3 



reachability 


CF NGCSS : 


NGCE i : 


I 


NCCE c : 


1 


NCCE 3 : 


A 


NCCE A : 


2 


NCCE 5 : 


A 


NCCE 6 : 


A 


SLY : 


u.coccc 


R SACHA 8 IL IT Y 


INC EX 


Cr OIRECTEC 


GRAPH: 
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ANNEX G 



Psae 2 0 of 54 



DIRECTED G»AP^ REPRESENTATION 

PROJECT a : a 

Program cart : LISTSHCPT 



COMPLEXITY MEASURES: 



NUMBER OF STATEMENTS: 13 

NUMBER CF NCCES: 3 

NUMBER CF ARCS : 3 

MJ N B E R CF PATHS: 3 

CYCLCYATIC NUMBER: V ( G ) * 



2 



REACHABILITY 



NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 

NCCE 



SUM : 

REACHAe IL IT Y 
Cr DIRECTEC G 



C r NCCES: 
1 
1 
1 
1 
1 

1 
1 

2 



9.C0CCC0 
INC EX 
R APH : 



i . 12S00C 
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ANNEX G 



Pace 21 c* i* 



DIRECTED GPAPn REPRESENTATION 



PROJECT « : u 

Program cart : FIND v E M B E R , F I NDA T r R IdUTE , aRTTE 



COMPLEXITY MEASURES: 



NUMBER OF STATEMENTS: 13 

NUMBER CF NCCES: 7 

NUMBER CF ARCS : 3 

NUMBER CF PATHS: 7 

CYCLCPATIC NUMBER: V(C)= 



1 o 



3 



RtACFAS IL 1 7 Y CF 



NCCE 
NCCE 
.NCCE 
.NCCE 
NCCE 
NCCE 
NC CE 



NCCES : 
1 
7 



fc 

7 

7 



SO ^ • 34.CCCC0 

REACHA5 IL I7Y INCcX 
CF 01 R EC TEC GRAPH : 



4.S57142 




- 2 q (in 

P e s u 1 ts") 



RESULTS 
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ANMfcX r. 



Pace ZZ o f 3 ^ 



DIRFCTED Grt'APH 

PROJECT - : a 

Program cart : L ( S 1 * T T p I PU TE 

a) 3 of ncoes: 16 

b) a of arcs : Z 1 

c) 3 of statements: 3 'J 
a) 3 of cams: * 

e) 9 e ac r ability: * 

f) Cyclomat 1C n y -i n e r : 7 

* 'juoirer of earns 

and reachability 
are yo-y laroe. 



2 E p PF.5E^TAT IGTi 
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NNEX G 



Pace 23 of 3a 



. DIRECTED GRAPH REPRESENTATION* 
PROJECT a : a 

Program cart : LIST' 1 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: 2a 
NUMBER CF NCCES: 13 

NUMBER CF ARCS : 16 

NU >8 £R CF PATHS: 13 

CYCLCHATIC NUH9EF : V ( G ) = 5 

REACHABILITY CF NGC E $ : 

1 
1 
2 
2 
2 
2 
2 

a 

A 
6 
8 

16 



SI.*: 6C.CCCCC 

REACHA8 IL IT V INCcX 
CF OIRECTcC GRAPH: 4.615334 



NCCE 


2 


NCCE 


~ 


NC C E 


4 


NCCE 


5 


NCCE 


6 


NCCE 


7 


NCCE 


E 


NCCE 


9 


NCCE 


1C 


NCCE 


11 


NCCE 
\r r? 


12 
] " 




P4 
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A f-j Nl £ X Vi 



Pace 2 u of 3<i 



DIRECTED GKAph PEPPESE'jT AT ION 

PROJECT - : a 

Program oart : COMP APE PA*if,£ 



COMPLEXITY V '£A$UHE5: 

number OF SI ATE^ENTSs 19 
NUMBER CF NCCES: 19 

NUMBER CF ARCS : 24 

NUMBER CF PATES: 16 

CYCLCMATIC NLMEER; V { G ) = 



REACHABILITY CF NCCES: 
1 
1 
I 
1 
l 
1 
1 



A 

A 

A 

A 

A 

u 

c 

S 

16 

16 



83.00000 
REACHAE TL ITY IiNCEX 
CF D IRECTEC GRAPH : A 



NCCE 


l 


NCCE 


2 


NCCE 


•3 


NCCE 


A 


NCCE 


e 


NCCE 


6 


NCCE 


7 


NCCE 


3 


NCCE 


9 


NCCE 


LC 


NCCE 


11 


NCCE 


12 


NCCE 


12 


NCCE 


14 


NCCE 


15 


NCCE 


16 


NCCE 


17 


NCCE 


18 


NCCE 


19 



SUM : 




301 




ANNEX G 



P aqe 2 S o ^ 5u 



DIRECTED GRAP^ PEPHESENTA r IOM 

PROJECT a : a 

Proqraf” oart : ATTR, V A TCP IMG/ CHANGE *EY 



C0 M PLEx I T Y MEASURES: 

DUMBER OF STATEMENTS: 10 
NUMBER CF NCC6S: 6 

NUMBER CF ARCS : 6 

NUMBER CF PATFS: ' 2 

CYCLCF AT IC NUMBER : V (G ) = 



REACFAS II ITV 
NC CE 1 : 

NO CE 2 : 

NC C E 2 : 

NG CE A : 

NC C 3 5 : 

NC C E 6 : 



CF NCCES: 
1 
1 

1 
1 

2 



sum: a.cccooo 

RE AC FACILITY INCEX 
CF OIRECTEC GRAFF : 1 




-j •= 23 - 
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ANNEX G 



Pace 2 b at 3a 



DIRECTED GRAPh representation 

PROJECT 5 : a 

Proara'n oart : r A 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: ir 
NUMBER CF NCCcS: IQ 

NUMBER CF ARCS : 11 

NUNBER CF PATHS: 2 

CYCLCMATIC NUMBER: V (G)= 3 



REACHABILITY CF NCCE3: 
NCCE l : L 



NCCc 


l 


1 


NCCE 


2 


1 


NC Cc 


3 


l 


NCCE 


2 


1 


NCCE 


5 


I 


NCCE 


6 


1 


NCCE 


7 


1 


NCCE 


c 


2 


NCCE 


9 


3 


NCCE 


L 3 


9 




SUM : 


15.COCCO 


REAChAB IL ITY INCEX 


CF 0 


IRECTEC 


GRAPH : 
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ANNEX G 



Page 2 7 of 3 4 



DIRECTED GRAPH REPRF. SENT A T 10 i 

PROJECT s ; a 

Program oart : FIND 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: 1* 
NUMBER CF NCC'ES: 11 

NUMBER CF ARCS : 13 

NUMBER CF PATHS: 5 

C YCL CM AT IC NUMBS c : V(G) = 



R EACHAe IL I TV 



NCCE 
NCCE 
NCCS 
NGCr 
NCCE 
NCCE 
NCCE 
NO C E 
NOCE 
NCCE 
NCCE 



1 



c 

9 

1C 

11 



CF NCCES: 
1 

1 
1 

2 
1 
2 
2 
2 
2 
A 



SUM: 22.CCCC0 

R E AC H A E I L I T Y INlcX 
CF CIRcCTEC GRAPH: 2 



O 




PS 



32 
3 3 





ANNEX G 



P age 28 o + 3 4 



0 F P E C f £ 0 GPiPH »fc°PESENTA T IQN 

PROJECT tt ; a 

Program o a r r : GET P E C U £ 3 T 
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ANNEX G 



Pace 2 q or 54 



DIRECTED G RaPm D E P R E 5 E N T A f I 0 N 

PROJECT s : a 

°raqram oart : E *3 U A L 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: JR 
NUMBER CF NCCES: 27 

NUMBER* CF ARCS : 

NUMBER 'CF PATHS; 321 
CYCLCMAT IC ‘NUMBER : V ( G ) = 



Error 4 50 




SsiOASIlITY 


CF NCC 


NC CE 


1 


1 ’ 


NCCE 


2 


1 


NC CE 


■2 


1 


NCCE 


4 


1 


NCCE 


c 


2 


NCCE 


c 


1 


NCCE 


7 


2 


NCCE 


a 


2 


NCCE 


9 


14 


NCCE 


1C 


12 


NC L c 


1 1 


6 


NCCE 


12 


12 


NCCE 


13 


12 


NCCE 


14 


14 


NCCE 


15 


14 


NC Cc 


16 


2 S 


NOCE 


17 


- 2 


NCCE 


17 


c- 


NCCE 


IS 


32 


NCCE 


19 


6 4 


NCCE 


2 C 


64 * 


NCCE 


21 


6 4 


NO CE 


22 


12 9 


NCCE 


5 2 


IcC 


NCCE 


2 A 


16 C 


NCCE 


25 


323 


NCC: 


26 


321 



SUM : 1469 *00 0 

RE AC PABILITY INCSX 
CF CIRECTEC GRAPH: 54,3703c 



47 
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ANNfcX G 



P aae 30 of 34 



DIRECTED GRAPH °E°RESE’'iTAT ion 

PROJECT 3 : a 

Program cart : TEST CO fJ DITTO NS 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: 19 

NUMBER CF NCC6S: 10 

NUMBER CF ARCS 2 12 

MJY8ER CF PATHS: 6 

CYCLCYATIC NLY6ER: V(G) = 

REACHABILITY CF NCCcSs 

1 

2 
1 
2 
2 
2 
2 
2 
A 
E 

2A.C0000 
REACHABILITY IMCEX 
CF DIRECTED GRAPH: 2 . AC COCO 



NCCE 

NCOS 

NCCE 

NGCE 

NCCE 

.NCCE 

NCCE 

NCCE 

NCCE 

NCCE 



1 

2 

5 

c 

6 
7 
a 
9 

IC 



SLY : 
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ANNEX G 



Page 31 of 3a 



DIRECTED GRAPH REPRESENTATION 

PROJECT ft : (X 

Program part : COMPARE MANY 



a) ft of nobles: 12 

b) ft of arcs : la 

c) ft of stat emen t s : 

d) ft of oaths: * 

e ) Reac h a b i 1 i t y : * 
f) Cyclomatic number 
* Numbe r of paths 
and r eac h ab i 1 i t y 
are very 1 a rae . 
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ANNE< C, 



Pace 32 of 3^ 



DIRECTED GRAPH r A T IQfj 

PROJECT s : a 

Program oart : LII^I T 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: 15 
NUMBER CF NCCES: 7 ' 

NUMBER CF ARCS : d 

NU^ER CF FAThS: 7 

CYCLCMAT IC NUMBER : V(G)= 3 



REACHASIlITY CF NODES: 



NO Cc I : I 

NCCE 2 : 7 

NC Cc 3 : z 

NCCE 4 : 3 

NCCE 5 : 6 

NCCE 6 : 7 

NGCE 7 : 1 



SLY: 26 • CCCGO 

RE ACFA6 II I7Y INCEX 
C F DIRECTEC GFAFh: 5.142357 
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ANNEX G 



Paae 33 of 3 ^ 



OltfECTEV' CP4PH J£PPESEM AT ION 

PROJECT 3 ; a 
Program oart : U L I M I T 



COMPLEXITY MEASURES: 

NUMBER OF STATEMENTS: 1 7 

NUMBER CF NCCES: 1C 

NUMBER CF ARCS : 12 

NUMBER CF PAThS: 14 

CVCLCMATIC NUMBER : V(G)= 4 



R5AC1-ABIL ITY CF NCCES: 


NCCE 1 


i 


NCCE 2 


2 


tNCCE i 


1 


NCCE 4 


2 


NCCE 5 


14 


NCCE 6 


1C 


NCCE 7 


e 


NCCE 6 


12 


NCCE 9 


14 


NCCE 1C 


14 


SUM : 


76, CCCCO 



REAChAB IL ITY INC £ X . 

CF C IF EC TEC GPAPh: • 7.599999 
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ANNEX G 



Paqe 3u of 



OI^ECTEO GRAPH ^PRESENTATION 



PROJECT = : a 



Rrogram oarr : 3 I T C H 



COMPLEXITY MEASURES: 

NUMBER* OF STATEMENTS: 15 
NUMBER CF NCC £ S : 4 

NUMBER CF iRCS : 4 

NUMBER CF PATHS: 2 

CYCLCMATIC NUMBER: V(G) = 



R5ACHA6 ILITY CF NCOES: 
NCC S 1 : 1 

\CCE 2 : i 

NCOS 3 : 1 

N C C E 4 : 2 




SUM : 5.CCOOCO 

R E AC FA 0 It ITY INCEX 
CF CIRECT5C GRAPH : 1.25CCOO 
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ANNEX H 



Page 1 



of 3 



TEST PHASE DESCRIPTION 

Project A : 4 

Test run & 2 1 Including 5 Test Steps 

Begin of Test (day/time) : 00/29/1330 End of Test (aay/time) : 0a/29/l53G 

CPU time for necessary compiles (in sec.): 9.09 

a) 9.09 b) c) d) e) f) g) 

CPU time for TEST run (sec) : 12.8 

Man Hours for this Test run : 2.0 (including preparation of 
tests) 



1 ) 



TEST 

STEP 


OBJECT VE J EXPECTED 

! RESULT 
I (TOLERANCE) 

i 

i 


ACTUALJERRQR 
RESULT! a 

i 

• 

• 


1 —i O 

1 *-i J> 

» -< 
1 fT\ 


COMMENTS 

AND 

CODED ERROR 
TYPES 


1 {Test of I/O f unc- 51/0 can be direc- 

lions under CP/CMS 'ted as designed: 


o 

• 

4 


a/29 

1330 


1 ) Record when 
error occurs 



-Da t a base infor- 
mation is read 
from file laDeled 
"DBASE INPUT" 



2 



3 



Check Boundary 
Condi t i ons under 
CP/CMS 

Test SU f unc t i ons 
under CP/CMS 



-Commands are 
input via termi- 
nal only 
-Output can be 
directed oy user 
to terminal or 
file 

Program neglects 
incut or provides 
appropriate error 
messages 

a)MemOers can be 
added to the data 
base 

b ) Da t a base in- 
formation can be 
can be written on 
output file such 
that this file 
coul d be used as 
input file for 
subsequent runs 
c) Items of M 2 can 
be e x am i nee 



O.K. 



O.K. 



0 ! Chec k D i agnost i c s 

5 {Test Termination 
|of program 



O.K. 

O.K. 



a/29 

1550 
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ANNEX H 



Page 2 



of 3 



TEST PHASE DESCRIPTION 



Project * : 4 

Test run # : 2 Including 2 Test Steps 

Begin of Test (day/time) • 05/01/1200 End of Test (day/time) : 05/02/2200 

CPU tine for necessary compiles (in sec.): 8.75 
a)8.75b) c) d) e) f) g) 

CPU time for TEST run (sec) : 12.40 

Man Hours for this Test run : 4.0 (including creparation of 
tests) 



TEST 

STEP 


QSJECTVE 


EXPECTED 

RESULT 

(TOLERANCE) 


ACTUAL 

RESULT 


1 ) 

ERROR 

u 


DAY 

TIME 


COMMENTS 

AND 

CODED ERROR 
TYPES 


1 


Test comoar i son 


Comoa r i son must 


Does 




5/01 


1 ) Rec ord when 




of range values 


a 1 ways be correct 


not 




1200 


error occurs 








wo r V 




1700 


03 








for 






Imp 1 emen t a t i on 








nega- 






of new code 








tive 






nec es s ary to 








num- 






handle all 








bers 




5/01 


cases 












1730 




2 


Receat step 1 




O.K. 




5/02 






using corrected 








1600 






version of program 








5/02 














2200 
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ANNEX H 



Page 3 



of 3 



TEST PHASE DESCRIPTION 



Project * : 9 

Test run A : 3 Including 1 Test SteDs 

Begin of Test (day/tine) 2 05/03/0900 End of Test (day/time) : 05/03/1800 

CPU time for necessary compiles (in sec.): 9*68 

a) 9*68 b) c) d) e) f) g) 

CPU time for TEST run (sec) : 12.8a 

Van Hours for this Test run : 5.0 (including oreoaration of 
tests) 



1 ) 



TEST 

STEP 


OBJECTVE 


! EXPECTED 
J PESULT 
J (TOLERANCE) 

i 

i 


5 ACTUAL {ERROR 
'RESULT,' * 

• i 
i i 
■ i 
i i 


DAY 

TIME 


COMMENTS 

AND 

COOED ERROR 
TYPES 


1 


Test program unaer ! A! 1 functions 


i » 

i i 


5/03 


t ) Record when 




CP/C using 


{should work the 


• i 

i t 


OR 0 0 


error occurs 




radars as data 


! same way as for 


! O.K. { 








member s 


! members used i n 


i i 

i i 










{ previous test and! ' 










ideQug runs 


i i 

i i 










! ( orov i ded that 


• i 

* i 










Jail attribute 


i i 

• i 










lvalue pai rs are 


i i 

• i 










{within des i gned 


• i 

i i 


5/03 








{limits) 

i 

i 


• i 

i i 

i i 

i i 


1800 
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